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Abstract 




Enclosed is the final report for The Sound-to-Speech Translation System 
Utilizing Graphic Mediation Interface for Students with Severe Handicaps 
Project, U.S. Dept, of Education Grant No. H180P90015, submitted to the U.S. 
Dept, of Education, Division of Technology, Educational Media and Materials. 
The Sound-to-Speech Translation System Utilizing Graphic Mediation Interface 
for Students with Severe Handicaps (STS/Graphics) system is a voice 
recognition computer-based system designed to allow individuals with mental 
retardation and/or severe physical disabilities to communicate with others and 
to control their environment. This includes individuals with quadriplegia, spinal 
cord injury, cerebral palsy, aging problems, arthritis, and assorted neurological 
disorders. Operation of the STS/Graphics System is through vocalization, 
switch closure, or keyboard input for activation of electrical appliances and/or 
digitized speech output. Choices available to the user for system operation are 
presented as customized photographic-quality symbols representing familiar 
items/persons in the user's environment and are displayed on a monitor. The 
user makes choices through direct selection, linear scanning, or row/column 
scanning. 



This research project had four major outcomes: 1) development and refinement 
of the STS/Graphics system; 2) application, evaluation, field testing, and 
research results of the STS/Graphics system with students having mental 
retardation; 3) development of training strategies and a training manual on the 
use of the STS/Graphics system; and, 4) documentation of the STS/Graphics 
system research with two videotapes: Voices of the Future and the Operation 
of the STS/Graphics System. 



The STS/Graphics system represents a powerful and versatile means by which 
children and adults with severe disabilities can increase their independence and 
productivity. Assessment and educational applications are additional uses of 
the system. This research project is a major step towards the use of speech 
recognition as a natural means of communication, learning, training, and 
computer access by students having mental retardation and other disabilities. 
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A Comparison of Actual Accomplishments 
to the Goals Established for the Period 



The Sound-to-Speech Translation System Utilizing Graphic Mediation Interface 
for Students with Severe Handicaps project initially was funded as a 15 month 
project. A no-cost two month extension was granted making it a 17 month 
project. Each goal and objective of the project is listed below is specifically 
addressed. For some Objectives, there is more detailed information available in 
other Sections of this report. 

Goal 1: Outline, modularize, develop and configure the design of the 

STS/Graphics system software. 

Objective 1_ Design the software. 

This objective was completed. See the Software/Hardware Section. 

Objective 2 Program the STS/Graphics system software. 

This objective was completed. The Software/Hardware Section contains 
photographs of the STS/Graphics system software screens. The attached 
videotape Operation of the STS/Graphics System documents the operation of 
the system. 

Objective 3 Assemble the STS/Graphics system 
hardware and development software. 

This objective was completed. The Software/Hardware Section contains 
photographs with description of the STS/Graphics system hardware. 

Objective 4 Alpha Test the STS/Graphics system. 

This objective was completed in the Bioengineering Program laboratory. 

Objective 5 Document the STS/Graphics system code. 

This objective was completed. The Software/Hardware Section contains 
examples of the software code which has documentation done by the 
programmers. The software has been copyrighted with the U.S. Copyright 
Office. 

Goal 2: Conduct the technical and clinical evaluation of the STS/Graphics 

system. 

Objective 6 Identify Subjects. 

This objective was completed. Specific information on the subjects is found in 
the Methodology Section. Evaluation documents are in the Evaluation Section. 
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Objective 7 Beta Test the STS/Graphics system. 

This objective was completed. Specific information on beta testing is included 
in the Methodology Section of this report. 

Objective 8 Refine the STS/Graphics system. 

This objective was completed. Refinement was ongoing throughout the 
project. 

Goal 3: Analyze the data and write reports. 

Objective 9 Analyze the data. 

This objective was completed. Analysis of the data is found in the 
Methodology Section of this report. The publications and reports of this project 
are listed and contained in the Publication Section. 

Objective 1_0 Evaluate the STS/Graphics system hardware. 

This objective was completed. Evaluation of the STS/Graphics system was 
ongoing and specific information can be found in the Software/Hardware 
Section of this report. 

Objective 7 7 Develop the Training Manual. 

This objective was completed. The Training Manual can be found in the 
Training Manual Section of this report. 

Goal 4: Disseminate Information 

Objective 1_2 Develop a documentary videotape. 

This objective was completed. Two videotapes, Voices of the Future and 
Operation of the STS/Graphics System, are included with this report. Voices of 
the Future is a documentary of the research process. Operation of the 
STS/Graphics System documents the operation and setup of the STS/Graphics 
system. 



Objective 1_3 Disseminate information. 

This objective was completed. The articles which were written and the 
professional presentations regarding this research are found in the Publication 
Section. It is the intent of the research to submit the research findings of this 
research to the journal of AAC: Augmentative and Alternative Communication 
for publication in the immediate future. 

A major point of information is that the STS/Graphics system received 
recognition as one of the top eleven national finalists in the Johns Hopkins 
National Search for Computing Applications to Assist Persons with Disabilities, 
1992. The American Speech-Language-Hearing Association solicited a 
presentation of this research at their 1992 national convention. 



Page 2 of Objectives 



STS/Graphics 



Grant No. H180P90015 



Goal 5: Market the STS/G Software/Hardware 

Objective 14 Involve Marketers with the project. 

This objective was not completed. Details of the efforts made to involve a 
manufacturer and reasons for lack of success are as follows: 

The Arc approached multiple vendors of either augmentative communication 
products or Software/Hardware publishers in the assistive technology field 
regarding the STS/Graphics system. Companies contacted and outcomes were 
as follows: 

Prentke Romich (PR) : With travel money from this project, in February of 
1991, project staff travelled to Wooster, Ohio, and demonstrated the 
STS/Graphics system to Prentke Romich staff. Prentke Romich decided that 
the Graphics interface and the clinical and training applications of the system 
were extremely powerful. However, since PR develops stand alone units, it did 
not think that the color graphics approach would work with their products. 

They had also recently invested heavily in expanding their operation and in 
developing some new products. They did not have the capital to invest in new 
products, especially anything that was not in keeping with their product line. 

Apple Computer, Inc. : The STS/G system was demonstrated to Mr. Gary 
Moulton of Apple Computer, Inc. He was very impressed with all components 
of the system, especially its customizability, speech recognition and training 
potential. He said that Apple Computer did not have the resources to convert 
the system from a PC platform to a Macintosh platform but he helped facilitate 
extensive discussions with Don Johnston Development Company. 

Don Johnston Development Company : The STS/Graphics system was 
demonstrated to Don Johnston and his staff in October, 1992. The company 
had just invested heavily in Ke:nx and had just released it to the public. An 
additional problem was that most of the Don Johnston products are developed 
on a Macintosh platform and the STS/G system is on a PC platform. There 
have been subsequent discussions about obtaining Small Business Innovation 
Research funds with Don Johnston to change the system from a PC platform to 
a Macintosh platform. Nothing further has been done to date. 

Johns Hopkins National Search for Computing Applications to Assist Persons 
with Disabilities : Technology transfer and linking manufacturers/publishers 
with inventors was a stated outcome of the Johns Hopkins National Search. 

As a national finalist and winner in this competition, project staff attempted to 
take advantage of the technology transfer opportunities that were presented. 

In actuality, there were few opportunities for realistic discussions about 
technology transfer so this opportunity did not result in the STS/Graphics 
system coming to market. 

Laureate Learning Systems : Mary Wilson of Laureate Learning Systems was 
impressed with the potential of the STS/Graphics system. However, it was not 
within the product line or the hardware platform of Laureate Learning Systems 
products. She connected The Arc with someone interested in technology 
transfer but it never came to fruition. 
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products. She connected The Arc with someone interested in technology 
transfer but it never came to fruition. 

Access Unlimited of Houston. Texas : Access Unlimited was not able to 
support the STS/Graphics system marketing and distribution, but it was willing 
to offer the service of creating the photographic libraries for buyers of the 
system. 

Baylor Biomedical Services : BBS carefully evaluated the STS/Graphics system. 
After much deliberation, Baylor decided that it did not have adequate financial 
reserves to bring the system to market in the immediate future. 



Additional funding for this project was 
received by: 

The Arc Research Fund 
Martin Marietta Aerospace 
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METHOD 



Rationale 

Current research recognizes the power of using familiar items/activities within 
natural teaching environments/sequences for persons with mental retardation. 

A focus of this research is to determine the ability of the subjects to make 
meaningful choices of preferred and familiar items in their environment. After 
the ability to use voice for selection purposes was established, the researchers 
extended the application of the STS/Graphics system to novel items and their 
picture representations. By initially starting with familiar items, researchers 
were not training the subjects for cognitive and language understanding about 
the meaning of new objects or photographs. But more importantly, after 
learning how to operate the system, the subject was immediately able to 
communicate about an item that s/he knew about and liked. By including items 
from participants' natural environment on the monitor display, it was felt that 
their performance and ability to generalize the concepts would be enhanced. 

For these reasons, the items used with the subjects during the subject selection 
and evaluation phases were included among the items used during the training 
and intervention phases of the research project. The STS/Graphics Project 
operated on these premises in designing, and teaching participants to use, the 
system. 

Additionally, by enabling children to use their own voice as well as their own 
personal "speech" referents for the items, their ability to operate the system 
would again be enhanced. The STS/Graphics system was designed to provide 
enhanced opportunities for independence through communication and 
environmental control by acting as a vehicle for speech translation and action. 



Potential Subject Selection 

Subjects were selected from the schools in the Dallas/Fort Worth area. Initial 
evaluations were made in the Hurst-Euless-Bedford (H-E-B) Independent School 
District and the Irving Independent School District. H-E-B has 18,768 students 
in 28 elementary schools, 5 junior high schools and 2 high schools. With 
1,909 students in special education, there are 83 students who are identified 
as having mental retardation, with 48 in the elementary schools, 17 in the 
junior high schools, and 18 in the high schools. Irving Independent School 
District has 25,064 students in 17 elementary schools, 9 junior high schools 
and 3 high schools. With 2,727 students in special education, there are 187 
identified as having mental retardation with 126 in elementary school, 30 in 
junior high school and 31 in high school. The 6 subjects selected for final 
participation in this research were evenly split 3/3 from each school district. 

School administrators were contacted in order to assist the STS/Graphics 
project team in the identification of potential subjects. Letters were sent to 
school administration describing the project and subject selection criteria. 
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School administrators identified teachers in their school systems who could 
more effectively identify potential subjects. Interviews were conducted with 
relevant school staff to assist in this determination. 

Letters were sent to the parents of potential participants, whose names were 
derived from the previous activities, explaining that their child was being 
considered for the STS/Graphics project. This was followed by brief interviews 
with the parents who responded to the letter to determine whether their child 
would meet some of the named criteria. If parent descriptions suggested that 
the child may meet the criteria, parents were asked to sign a release of 
information form for the school to enable the project staff to review school 
records. This was necessary to help determine whether or not the child had 
mental retardation and testing and performance scores were on file as 
documentation. Other relevant information was recorded such as medical 
information, etc. which suggested a student's performance and behavior in 
school, school attendance, and parent support. 

Criteria for Subject Selection 

Criteria for the selection of the subjects in the STS/Graphics research project 
included: 

o adequate vision and hearing 
o adequate positioning and seating 
o understanding of picture/object association 
o understand cause/effect relationships 
o rudimentary scanning skills (if appropriate) 
o speech is not functional as a primary mode of communication; 

i.e., the subject used unintelligible vocalizations 
o intellectual and adaptive behavior measures that classified the 
student as mentally retarded 

Final Subject Evaluation 

There were two phases to the subject evaluation. The first phase was based 
on parent/caregiver/teacher interviews. The second phase was based on direct 
evaluation of the children. 

Phase 1: Parent/Careaiver/Teacher Interviews 

Following the initial parent interview and record review described above, 24 
students (12 male and 12 female) were selected for further evaluation. 

Potential subjects were evaluated ranging in ages from 4-21 years. Twelve 
attended school in the H-E-B school district and 12 attended school in the 
Irving school district. Diagnoses included cerebral palsy, autism, encephalitis, 
Down's syndrome, and renal dysfunction. Subjects varied level of mental 
retardation from mild to profound. Of the 24 subjects evaluated, 14 were 
ambulatory and 10 were non-ambulatory. 
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As part of this evaluation process, a more detailed interview was conducted 
with one or both parents/caregivers and school staff. The interviews provided 
the following information: 

1. Medical status with regard to overall health and medication; 

2. Sensory skills with regard to vision and hearing skills; 

3. Gross and fine motor skills; 

4. Seating and positioning considerations; 

5. Cognitive skills; 

6. Receptive and expressive language skills; 

7. Communication strategies; 

8. Previous experience with augmentative communication assistive 
technology; 

9. Preferred items in their home and school settings; and, 

10. Whether or not specific speech sounds were used consistently to 
reference these preferred items in their environment. 

Questions were conducted with familiar staff, family and peers to identify 
preferences, behaviors, and vocal patterns in the natural environment. 

Students were present in the interviews, but the main purpose of the 
interviews was to obtain information through observation and from others 
regarding student ability, skills and preferences. This information was recorded 
on the interview form by the project staff (see the Evaluation Section for the 
Interview Form). It was important that the subjects demonstrated that they 
had consistent and meaningful speech sounds. It was assumed that this ability 
would make it more likely that the subject would be able to make effective item 
selections from the STS/Graphics system. Parents were asked to bring some of 
the reported preferred items with them to the evaluation sd.they could be 
incorporated in the evaluation process. 

Research assistants wrote down the students' spontaneous utterances and 
imitations with their associated item. The interviews, record review, and 
observations provided the necessary information for the child's participation in 
the project. The information was used to plan later project activities if the 
student was selected as a STS/Graphics research subject. 
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Phase 2: Direct Evaluation of Potential Subjects 
Evaluation Procedure to Identify Beta Subjects . 

There were certain basic skills needed by any subject to be able to use the 
STS/Graphics system: 

1 ) picture/object association 

2) cause/effect relationships 

3) rudimentary scanning 

4) sound production 

5) definite preferences 

Picture/object association was necessary to insure that the subject understood 
that the item displayed on the computer monitor in the form of a photographed 
image represented the actual item. Cause/effect understanding was necessary 
to ensure that the subject understood that when making a vocal selection from 
the items presented, that the resulting action was caused by the vocalization. 
Scanning skills were necessary to ensure that the subject understood how to 
make a selection using a scanning routine, if direct selection was not possible. 
Sound production was necessary to ensure that the student could use his/her 
voice to control the STS/Graphics system. Demonstration of definite 
preferences ensured that the items made available to the subject were indeed 
high preference items for the subject so that there was more assurance that 
subject behavior was not influenced by items disliked by the subject. 

Potential subjects were assessed on their abilities to demonstrate each of these 
skills according to the following procedures: 



1) Picture/object association skills. There were two steps in the 
picture/object association evaluation: item (stimuli) selection and 
photograph preparation. 



Item (Stimuli) Selection . With the item/stimuli selection, researchers felt 
that the ability of the potential subjects to make choices from between 
two alternatives would be adequate for effectively operating the 
STS/Graphics system. Choosing between two items was the criteria for 
success in this part of the evaluation. For information purposes, some 
students were given the opportunity to make more than two choices to 
determine if they had this ability. Items were used which had been 
identified during the interview process and provided from home by the 
parents. Project staff provided additional items in the event that the 
reported item was not truly motivating to the child. 

The students were given two opportunities to make choices. First, two 
actual items were presented to the student by the researcher holding the 
items. In the first presentation, the student was asked to point to the 
item s/he wanted in an effort to identify the more highly preferred item. In 
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the second presentation, the student was asked to show the researcher 
with item which was named in an effort to ensure that the student could 
identify the item. If the student was capable, s/he would point to the 
preferred item. If unable to point, the student would eyegaze his/her 
choice. Once a selection had been made, the location of the items were 
reversed from either left to right or right to the left to ensure that the 
selection was meaningful. 

Photograph Preparation and Item Selection . Prior to the evaluation of a 
student, project staff asked parents and school staff to loan items to the 
project staff to photograph. Items were photographed in the 
Bioengineering lab with the items placed on a white background so that 
the surrounding background of each item photographed was consistent. 
Thus, the pictures used during the evaluation process were photographed 
to ensure clarity and freedom from visual background distractors in the 
photograph. The actual items were then returned to the school or home. 
The students were introduced to the photographs of the items utilizing the 
same procedure described in the Item/Stimuli Selection section above. 

On occasion, an intermediate step was used with a subject to assist in 
his/her ability to understand the selection process. The item would be 
presented beside the photograph of the item so that the student could 
make the association of the photograph representing the item. Ultimately, 
the student had to make the choice without the item present in order to 
become a research subject. 

This same procedure was used to introduce the images of the same items 
on the STS/Graphics system display monitor. First, with a full screen 
single image, then using one of the images in a 2X2 display (with 3 blank 
cells), and then two of the images in a 2X2 display (with 2 blank cells). 



2) Cause/Effect Skills. The students were given an opportunity to 
demonstrate their understanding of cause/effect by using a voice 
switch in several activities. All students were asked to use the 
voice switch regardless of physical ability. It was determined that 
they had cause/effect understanding when they independently 
operated the switch upon command from the parent or researcher 
(i.e., "make it go”, "make it stop") and then demonstrated control 
over this operation without any direction. Evaluation activities 
includes for use of the computer with cause/effect software 
programs; small appliances adapted for switches such as tape 
recorders; and, switch adapted toys. 



Cause and effect was evaluated using the voice switch both to play the /?. 
J. Cooper Early and Advanced Switch Games on an Apple lie computer 
and to turn a tape player on and off. Software games used included 
Noises, Numbers, and Building Shapes. Using an AbleNet Control Unit, a 
tape player for music feedback to the student was connected to the voice 
switch. Several times the researcher demonstrated how the tape player 
was turned on by producing a sound. After repeated demonstrations, the 
researcher asked the student to "Turn on the music". Once the student 
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turned on the tape player, the time-out function of the AbleNet Control 
Unit turned the music off after a short time. The process was then 
repeated. Success was measured by the subject successfully playing the 
games or operating the tape player. 



3) Rudimentary Scanning Skills. The method by which a student 
would control the STS/Graphics system, either scanning or direct 
selection, was decided by assessing the students' ability to 
produce multiple sounds which were spontaneously associated 
with specific items. Students who did not demonstrate this ability 
were evaluated for scanning access. Students who did 
demonstrate this ability were assigned to direct (voice) selection. 



For the children who required scanning as a means of control, scanning 
was evaluated after picture/object association and cause/effect 
evaluations were completed. The Apple computer was utilized for the 
scanning evaluation with a software program which closely resembled the 
"blank display" pages from the STS/Graphics system. Scanning was 
evaluated by using the Scan and Speak Software on an Apple lie 
computer. A single photograph was attached to the computer monitor 
and the scanning software was activated. The voice switch was used by 
the subject to halt the scan box. A single photograph of an item 
previously used during the evaluation was placed on the monitor display. 

A 2 x 2 display was used (with 3 blank cells). The photograph was 
placed on the monitor in either of the top two cells. The student was then 
asked to use the voice switch to stop the "box" on the picture. The 
photograph was then placed at other cell location on the display. Once a 
student demonstrated the ability to stop the scan box over a single 
picture, the same process was followed using two pictures on the 2X2 
display. The student needed to successfully place the box on the picture 
five times. 



4) Sound Production Skills. The preliminary evaluation of the 
students' ability for sound production was done in the initial 
interview. Research assistants wrote down the students' 
spontaneous utterances and imitations matched with their 
associated item. During this round of evaluation, students were 
first presented with an object and asked what it was. Project 
researchers observed the student's response to the presentation of 
the preferred item and noted the student's use of speech sounds 
to reference the items, as well as whether or not the speech 
sound was used consistently with each presentation of an item. 
The sound that the student produced was written down by a 
research assistant. This was repeated for several objects. The 
objects were presented again to verify that the sounds were used 
consistently with the specific object. 
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5) Definite Preferences. A list of items identified by caregivers 
and teachers to be of interest to the subjects during the interview 
process was compiled for each subject. A variety of the objects 
on these lists were collected and photographed. The photographs 
were used in the first five intervention sessions (described below) 
with a subject. It was determined at this time that it was 
important to rank the photographs according to the subject's order 
of preference. This prevented any failure by the subjects to use 
the system effectively due to lack of interest in the items 
presented because they were not preferred. After these five 
intervention sessions, intervention was interrupted to perform 
forced choices for the preferred images. In the forced choice 
procedure, the subjects were shown photographs of items that 
were assumed to be high preference items. These photographs 
were randomly placed in sets of two for presentation to the 
subject. The list of pairs was constructed such that no 
photograph appeared more times than any other in either the right 
or left position and no photograph would appear in the same 
location twice consecutively. Every photograph appeared with 
every other photograph one time. The subject was asked to 
indicate using their personal style which of the two photographs 
was his/her preference. After a choice, the subject was allowed 
to interact with the actual item for a few minutes. Forced choice 
was conducted for approximately 5 sessions for each subject with 
no intervention during that time. This interruption of interventions 
occurred for all subjects. 



Evaluation Outcomes 



In summary, students who were selected for participation in the STS/Graphics 
research had to demonstrate their ability to understand cause and effect, to 
perform sound production/imitation, to make picture/object association, and to 
indicate an ability to scan. Each subject used the STS/Graphics System in her 
school environment. 

Alpha Subjects 

Two subjects, who were severely physically impaired but not mentally retarded 
(and therefore did not meet criteria), were selected as alpha subjects. The 
alpha subjects participated in a variety of informal tests of the STS/Graphics 
system to provide feedback to researchers about the soundness of its design 
and operation. 

Beta Subjects 

Six students were selected for participation as beta subjects: four females and 
two males; chronological age ranges 7-21 years; diagnoses of cerebral palsy, 
encephalitis, mild to profound mental retardation; four were ambulatory (two 
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walked with assistance of a walker), and two were non-ambulatory. The mode 
for accessing the computer were direct selection for three subjects and 
scanning for three subjects. 

Before the project ended, three subjects remained in the study. Two were 
direct selectors and one was a scanner. Two of the original subjects were 
dropped from participation because of health problems. The third subject was 
dropped from participation because from the time of evaluation until 
training/intervention, her teacher began to use Touch Window technology with 
her in the classroom. Consequently, this subject learned to use her hands 
rather than her voice in operating the computer, even though our assessments 
indicated voice operation would be a more preferred access mode. 

Subject TL was a female from A.S. Johnston Elementary School in the Irving 
Independent School District. She was 1 1 years 4 months at the time of her 
first observational evaluation. Subject TL had severe mental retardation and a 
seizure disorder which was most predominant when the subject was tired. 
Medication controlled the number of both grand and petite mall seizures. 

Subject TL exhibited a fine motor tremor which was became more pronounced 
when she was nervous, upset or under stress. She was ambulatory. 

School records showed that at 10 years 4 months, Subject TL was evaluated 
and exhibited an age equivalency of 36 months. When scored on both the 
Stanford Binet and Vineland Scale, her score was 35 or below. Standard 
scores on the Woodcock- Johnson Psychoeducational Battery fell below 65 and 
receptive language skills fell within the 3 to 6 year level. An Audiometric 
Screening was attempted but was not completed. Parents and teachers 
indicated that her hearing and vision was within normal range. 

Communication . Subject TL used speech, signing and pointing to objects 
to communicate and to indicate her needs. Her mastery of sign language was 
inconsistent, and signing was becoming less functional as she was attempted 
to communicate using speech even though it had limited intelligibility. Subject 
TL utilized one and two word sequences which could be understood by parents 
and teachers. In the community, (restaurants and the mall) she was learning to 
use a communication system made of black and white photographs. 

Computer Experience . Subject TL used a Tandy computer at home and 
an Apple computer at school which she controlled by normal keyboard access. 
The school was utilizing computer games to teach Subject TL turn taking skills. 

Subject MP was a female from A.S. Johnston Elementary School in the Irving 
Independent School District. She was 1 1 years 7 months of age at the time 
she was evaluated for participation in this project. She had mild cerebral palsy 
and seizures which were controlled by medication. According to her school 
records. Subject MP was evaluated 9 months prior to this project and scored 
27 on the Vineland Social Maturity Scale. Subject MP was ambulatory and 
indicated self help skills at approximately the 48 month level. 
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Communication . Subject MP was non-vocal but produced a variety of 
unique sounds. She used a communication board for ordering food at 
McDonald's and could sign approximately 70 to 100 signs which she used 
spontaneously and upon request. When asked to indicate her desires, she 
would hesitate and then walk over to the item. If selection was not possible 
using this method, she would go through a list to indicate her choice. 

Computer Experience . School records indicated that Subject MP had 
been working on the computer even though no details were recorded. 

Subject DL was a female from The Career Center in the Hurst-Euless-Bedford 
Independent School district. The Career Center is a self-contained school for 
high school students for transitioning students from high school to 
employment. At the time she was first observed. Subject DL was 21 years 4 
months of age. She had cerebral palsy, fibromuscular Hyperplasia, vascular 
system constriction in the bilateral renal artery, hypertension and had 
experienced 5 strokes since age 18. Subject DL had balance problems and left 
side involvement. Additionally, she had seizures during sleep, mood swings 
and decreased endurance. Subject DL exhibited limited range of motion and 
motor skills. Even though she was ambulatory she utilized a wheel chair during 
exercise or for mobility over longer distances. She did not propel herself, but 
needed to be pushed. 

One month prior to the observation of Subject DL, she was assessed using the 
Leiter International Performance Scale where she indicated an IQ of 35 and an 
age equivalency of 2 years 1 month. The Wechsler Adult Intelligence Scale 
was administered but no results were recorded. 

Communication . Subject DL was non-verbal with severe expressive 
language delays. She used a clip board communication system with line 
drawings of daily requirements. She also used approximately 75 signs from 
spontaneous sign language that her parents attribute to her watching a 
television show that taught signing. 

Computer Experience . Subject DL had experience using an Apple IIG and 
Apple HE at school with basic academic programs. She had inconsistent 
success with the computer. During the STS/Graphics project. Subject DL was 
given an Apple HE. By projects' end, she had not started to actively use the 
Apple computer at home. 

Subjects Selected to Use the STS/Graphics System 

The method used to introduce the STS/Graphics system to the two subjects 
who controlled the system through direct selection (Subjects TL and MP) was 
different than the method used with the scanning Subject DL. They will be 
described separately. 

Subject TL was an 1 1 year old ambulatory female, with severe mental 
retardation, who used direct selection to control the system. The subject made 
few vocalizations, all of which were unintelligible. Subject TL held the clip 
microphone close to her own mouth because she did not like it attached to her 
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clothing. Once she had produced a sound, she would usually hand the 
microphone to the researcher to free her hands in order to interact with what 
she had requested. 

Subject Descriptions of Scanners . Subject 3 was a 21 year old semi- 
ambulatory female, with mental retardation, who needed extensive support, 
and who used scanning to control the system. 



Procedure 

Instrumentation 



Key features of the STS/Graphics System were as follows: (a) photographic- 
quality computer-generated images of items/persons in the subject's natural 
environment that were presented on a display to represent choices among 
spoken communications and environmental control activations; (b) vocalization 
access, whether intelligible or not, that were used to select choices from the 
image display; (c) physical switch activations and traditional keyboard inputs 
that were alternate access modes; (d) direct selection via voice and keyboard 
access modes; (e) linear scanning and row/column scanning via voice and 
switch input; (f) speech output that was in the form of digitized, easily- 
understood speech that was age and sex appropriate for the subject; and, (g) 
data on system use that was automatically recorded, thereby facilitating the 
development of appropriate training strategies. A more detailed description of 
the STS/Graphics System follows: 

The following list of hardware components represents the hardware in the 
STS/Graphics System. 

o International Business Machine (IBM) Personal Computer (PC) 
Advanced Technology (AT) or compatible with a 286 central 
processing unit (cpu) 

o 30 Megabyte (MB) of hard disk space 

o Super Vector Graphics Adaptor (VGA) monitor 

o Paradise Professional Video Adaptor 

o Votan 2000 speech recognition board 

o Image capturing device with color capability 

o X-10 Powerhouse and various wall modules 

o 640 Kilobyte (KB) of RAM 

o Floppy drive 

o Microsoft-Disk Operating System, version 3.30 (MS-DOS) or greater 

(The Software/Hardware Section gives detailed information about the 
STS/Graphics system hardware, software, training of voice templates, 
environmental control, sound-to-speech translation and voice template 
matching, and access). 
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Microphone Issues 



During the initial stages of field testing, difficulty was experienced in locating 
the microphone in a position that was not a distraction to the subjects and yet 
allowed utterances to be "recognized" by the system. Reaction to the 
microphone varied from subject to subject, but in all cases it was a distraction 
from the items displayed on the monitor. It was decided to use one of two 
microphones with the students, either the stand alone Votan microphone/ 
speaker unit, or a Realistic clip-on microphone, positioned in different locations 
for different subjects. 

Access Modes 



The STS/Graphics software provided two methods of accessing the system: 
direct selection and scanning. In the direct selection mode, a subject vocalized 
a different sound for each image that was presented on the display. The 
scanning selection mode was operated in two ways: linear scanning or 
row/column scanning. In either scan mode, the system required only a single 
vocalization to access all choices and the vocalization did not have to be 
consistently pronounced from one occurrence to the next. The vocalization 
served to stop the scanning sequence. Once the scan halts, the functions 
executed that were associated with the image on which the scan stopped (that 
is, speech output, environmental control, or both). Alternately, the scanning 
could be stopped by using a single switch. 

Voice Input for Training Speech Templates 

When operating the STS/Graphics system, the subject's voice functioned as a 
switch and the subject "trained" the computer to recognize by speaking the 
target vocalization into the microphone. This sound was recorded digitally on 
the hard disk. The training was repeated three times and each vocalization was 
recorded separately. Once the three trainings were stored in a subject-specific 
voice template, the STS/Graphics software analyzed the three recordings and 
averaged them. 

Creating Voice Templates for Direct Selectors 

To create voice templates for direct selectors (this software function has been 
redesigned), a photograph of the desired object was held close to the 
microphone, the student was prompted to speak. The direct selection 
prompting sequence was: 

1 . "Tell me about the ", or 

2. "Use your voice to tell me about ", or 

3. "Tell me about the " - while pointing and taping the appropriate 

object on the monitor, or 

4. Modelling of the sound by the researcher 
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The system's record function was activated, and the utterance produced by the 
student was recorded by the system. This process was done twice and then 
tested by repeating the process again with the voice matching function 
activated. If the third test utterance was within an acceptable range of the two 
recorded utterances, then the recordings were saved into the voice template. 

If the matching range was inadequate, then the training process was repeated 
until the match was acceptable. This process was done without the computer 
activating any speech output or environmental control action in response to the 
vocalizations during the voice training process. 

Creating Voice Templates for Scanners 

For scanning access to the system, the subject specific template was 
comprised of trainings for one unique utterance per subject. The scanners 
prompting sequence was: 

1 . "Show me the " 

2. "Show me the " - tap monitor in the correct cell when scan 

box is on that cell. 

3. The researcher would demonstrate how to stop the scan action by 
vocalizing at the correct time and halting the scan action over the 
displayed picture. 

The scan box started moving through the 2X2 matrix, the researcher pointed 
to the scan box and whispered "not yet" or "no", and sometimes shook her/his 
head "no". This procedure was repeated on each empty cell. When the scan 
box was on the picture, the researcher tapped the screen more actively and 
whispered "yes". Praise, clapping, smiles etc., reinforced each successful 
activation. 

Once all of the trainings were stored, the system checked to verify that the 
subject's target vocalization would successfully match one of the sounds 
stored in the voice template. If the match was not similar enough to the 
original trainings, then the subject would retrain the voice template for that 
particular utterance. 

When the subject was actually operating the system and producing a 
vocalization, the system checked to see if the vocalization achieves a "match" 
with the trained vocalizations stored in the voice template. In the event that 
there was a match, the system would acknowledge the vocalization by 
activating the appropriate speech output message or environmental control 
function; in the event of the vocalization not matching any of the trainings 
closely enough, the system would not respond. If the system did not respond, 
two optional corrective actions could be taken. First, the accuracy used by the 
system to match a vocalization to the stored voice trainings could be made less 
sensitive so that the system was less demanding in its matching requirements. 
Second, the subject could choose to repeat the training process to refine a 
vocalization stored in the template for better recognition accuracy. 
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Sound-to-Speech Translation and Expansion 

For communication purposes, each subject could access speech output 
messages stored in digitized files on the hard disk. Each message could consist 
of speech of up to 8 seconds in duration, thus permitting single words or 
complete sentences to be output. A model whose voice was age and sex 
appropriate for the subject recorded these messages into the system. The 
subject's often brief and unintelligible vocalization translated into intelligible 
speech and expanded into a phrase or complete sentence. For example, if a 
subject's vocalization for "water" approximated "wuh", the system could 
immediately output "Could / have a drink of water, please. " Message files for 
many different subjects could be stored and accessed via one computer 
system. 

Environmental Control 



Any voice (or other) input could be linked to the activation and deactivation of 
designated electrical devices. A single voice input could also be linked to any 
combination of speech outputs and device activations, e.g., the vocalization 
"pa" translated into "I'm going to watch TV now", and the television set turned 
on. 

Graphics Interface 

The system displayed on the video monitor photographic-quality images of 
objects, appliances, and people in the subject's environment. The photographs 
with the highest ranking items were scanned into digitized images so they 
could be presented on the STS/Graphics display. These images were 
associated with spoken communications, environmental control, or a 
combination of both. While all of the subjects in the research used a 2 X 2 
matrix of images, the present system's capability allowed the size of the 
images, as well as the number of images appearing on the display, to be 
customized to the subject. Displays ranged from a single image to a 5 x 5 
matrix of images. The graphics display could also "flip" to a second page of 
choices for each subject. The two pages of images permitted the configuration 
of a hierarchical system of choices for subjects who understand the concept of 
categorical clustering. 



Research Design 

Researchers investigated whether subjects with mental retardation and/or 
multiple disabilities could learn the cause/effect relationship between 
vocalization and function. If subjects learned the cause/effect relationship, 
could they then learn that different vocalizations are associated with different 
functions? Would the subjects exhibit increased communication with others as 
a result of system use? The viability of the graphics component as an option 
for this population using this type of communication/environmental control 
system was also an important issue. And finally, what aspects of the system, 
if any, needed further refinement? 
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Experimental Sessions 

Phase One: Training of the Subject on System Operation with a 1 X 1 matrix 

In the first phase of the research design, training on system operations was 
done using a photograph of the object; later the image was displayed on the 
monitor. Training strategies were taught to the subjects on how to operate the 
system using their personal vocalizations. During single-image trials, there was 
only one graphics image displayed on the monitor in a 1 X 1 matrix (meaning a 
single image filled the screen), and the corresponding speech output or device 
was activated. 

Phase Two: Introducing a 2 X 2 matrix with one image . 

In the second phase of the research design, additional graphics images were 
added to the display in a 2 X 2 matrix and their corresponding choices were 
available for activation. Subjects would advance from single-image displays to 
multiple-image displays when they successfully completed at least nine out of 
12 trials on each type of display. 

In both phase one and phase two, experimental trials consisted of the 
researcher orally requesting the subject to activate a specific device or 
communication phrase, e.g., "Tell me about the beads". A trial was scored as 
successful when the subject emitted the vocalization that corresponded with 
the requested device or communication and the computer system recognized it 
as such and activated the proper output. Three levels of prompting could be 
used with subjects to help them attend to the monitor display and to learn 
either the direct selection or scanning strategies. These prompts consisted of 
verbal instruction, tapping on the display, and/or modeling the subject's 
vocalization to aid the subject in learning to correctly vocalize and to control 
the system. 

Phase Three: Introducing 2x2 Matrix with Two Images 

Explanation to the Subject . The subject was informed at the beginning of the 
session that "We will be doing something a little different/new today. / want 
you to tell me about/find the pictures. Later, you can ask/Show me the thing 
you want to do. " 

These expectations were established early in the 2x2 display level of training. 

Selection of Items for Multi-item Display . Items were be included on this 
display which had the "strongest preference value" in the subjects' hierarchy. 
Since these items will be used continuously for the duration of the subject's 
participation in the study, careful consideration must be given to the selection 
of these items. Selection should be based on subject's original forced choice 
selection data as well as overall performance with the items during the 1x1 
and 2x2 single items array. 
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Selection of images included on the 2x2 display . Images available from the 
subjects' original "forced choice" lists were used to make decisions regarding 
the introduction of multiple images on the display. Those items which had the 
"strongest preference value" as demonstrated by the forced choice selection 
procedures conducted for each subject, were used to create the multiple item 
displays. For the purposes of the 2x2 display, four images were selected 
which were topically similar (i.e., beauty aids, leisure activities, snacks) and 
demonstrated the "strongest preference value". The two items demonstrating 
the "strongest preference value" were introduced on the 2x2 display first. In 
some cases, the item of strongest preference "overpowers" all other choices 
presented on the display. In this instance, the item should be introduced as the 
fourth item on the 2x2 display. 

The rationale for using this type of image selection for the 2 x 2 or other larger 
displays introduced later in the study, is based on the amount of time the 
subject will "experience" the item. Should the subject complete criterion 
successfully at each level of 2 x 2 display activities, the subject would be 
introduced to the next display size (i.e., 3 x 3). It is imperative that when the 
items used during the 2x2 display level of testing are incorporated on the 
larger display, that they retain their "preference value" throughout present and 
subsequential training phases. 

Definitions . In order to facilitate the reader's understanding of the following 
procedures, the following definitions are given: 



1) Set : since the STS/Graphics system equipment required time 
to regenerate each new display screen, subjects were given 
consecutive three trials with each display format, i.e., with the 
same images displayed in the same cells. This group of three trials 
is defined as a "set". The display was changed each time a "set" 
was presented to the subject. 



a. Experienced set: a set of trials to which the subject has previously 
experienced. For this portion of the study, sets contained within block 
two and three (defined below) are "experienced sets". 

b. Novel set: a set of trials which used display formats the subject has 
never seen i.e., the same images displayed in different cell locations. 



2) Block : a collection of sets. The order and configuration of the 
displays used in the block was pre determined. There are six 
possible display formats. The subject experienced only four of the 
six display formats. 



a. Experienced Block a block to which the subject has previously 

experienced. 

b. Novel Block: trials which used display formats the subject has never 

seen i.e., the same images displayed in different cell locations. 
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c. Generalization Block : This block contains a total of six sets of trials. 
Four of the six sets are "experienced sets." In other words, the 
subject has had the opportunity to experience and use these sets 
previously. The remaining two sets are "novel sets" which had never 
been presented previously to the subject. For this portion of the 
project, the generalization block was presented as part of the third 
(final) block administered to the subject. The block was composed of 
four experience sets and two novel sets. 

Display Format 



1) Display Construction — Each display was created using two 
photographic images. Each image was placed on the display as 
designated belO.w: 
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Displays FIVE and SIX will be designated as "generalization displays" and 
will not be administered prior to successful completion of BLOCK THREE! 
Presentation of displays should follow order. 

Criterion 



1) Block One -- Each subject was provided with the opportunity to 
demonstrate their control of the STS/Graphics System. The first 
test block was comprised of four different displays. The subject 
had to obtain 80% accuracy, or nine out of 12 trials for each item. 



If criterion was achieved, the subject was moved to the next level, a 
progression to a 2 x 2 matrix with three images. 



2) Blocks Two and Three - If criterion is not achieved, the subject 
will be exposed to blocks two and three. A criterion of 80% had 
to be achieved in block three in order for the subject to continue in 
the study. If 80% criterion was achieved, the third block of trials 
included the standard block three in addition to the generalized 
blocks. 



Presentation Procedure . Operational Competence 
Time: 30 minutes 



1) Presentation order - The order which images are presented to 
the subject was determined by a list containing random ordering of 
possible two image pairings. The total trials for each block of 
trials (i.e., 1, 2 and 3) was 24 with the exception of the 
generalization block for block 3 (an additional 12 trials was added 
at the successful completion of block 3). The specified order of 
presentation was randomized for use with each subject. 



a. Prior to the use of any prompting sequences, the initial session should 
be completed using only statement l.a. below. It was videotaped and 
maintained as baseline information. 



2) Videotaping — For this portion of the training session, the 
videotape footage captured the affective behavior of the subject 
and the researcher or interactant. The camera was moved from 
the subject profile shot, which included the display, to capture the 
interactions (faces) of the subject and those interacting with them 
on the system. 



3) Procedure - The subject was provided with a two-minute 
waiting period before presenting the next level prompt. 
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a. The subject was told , "Now you can use the computer to ask for what 
you want . " 

b. If there was no response to this statement, the researcher should ask, 

"Do you want some or ?". 

If a successful activation occurs, then return to 3. a. Prompt the subject if 
necessary. 

c. If no response to the researchers statement in 3.b., the research 

assistant then said, "How would you tell me you want the or 

? ". 



If a successful activation occurs, then return to 3. a. Prompt the subject if 
necessary. 

d. If there was no response to statement 3.C., the researcher said, "How 

would you tell me you want ?". This was used with one item 

and used with the second item if there is no response. 

If a successful activation occurs, then return to 3. a. Prompt the subject if 
necessary. 

e. If there was no response to statement 3.d., the researchers modeled 
the communicative behavior using the subject's display. Researcher 
one used the system using the subject's target sound or scanning 
motion to communicate the preferred item to Researcher 2 as follows: 

If a successful activation occurs, then return to 3. a. Prompt the subject if 
necessary. 

Researcher 1 : "I'm going to try it. " The researcher activated system 

using the subject's direct select sound or scanning motion. 

Researcher 2: Acknowledged the request by providing the activity or 

item. In addition s/he stated, "Here is the . " 

f. The subject was prompted using the statement from 3. a. If there was 

no response, the researcher said, "Let's try together. " 

If a successful activation occurs, then return to 3. a. Prompt the subject if 
necessary. 

g. If there is no response by the subject to statement 3.f., the activity 
was discontinued and noted on the data sheet. 

Scanning Access 

The subject was positioned in front of the monitor/microphone, the picture was 

displayed, and the subject was asked to "tell me about the ". Whatever 

sound was vocalized was recorded by the computer. Only one sound was 
recorded and it became the switch to stop the scanning process. 
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It was not necessary for the subject to produce a specific sound initially 
because the STS/Graphics system was used with a very "wide" recognition 
range, therefore it would activate with any sound the subject made. As the 
subjects became more familiar with the system, they produced a more 
consistent sound and so that the recognition range of the speech recognizer 
was narrowed. During the initial phase, the room was very quiet while the 
computer was active since background noise could activate the system. 

Corrective Procedures for System Malfunctioning 

Within each of the phases described above, systems occasionally occurred. 
When these malfunctions occurred, it was necessary to do a check of the 
system for the integrity of the voice recognition function. When those checks 
were necessary, the following procedure was observed. Each subject was 
seated in front of the monitor and microphone and the full screen display of an 
image was shown on the monitor. The researcher prompted the subject "tell 

me about the "then pressed ENTER. The next sound that the computer 

"heard” was recorded by the computer and the machine was ready to repeat 
the sequence. After two trials of these recordings, the software sequenced 
automatically to the test mode. 

A test step was introduced to test the sounds of direct selectors because of 
ongoing hardware problems with multiple sounds. These problems occurred 
when a third unique sound was introduced into the voice template for a 
subject. In the test mode, the subject was prompted to repeat a sound, the 
machine recorded it and then compared the sound to the two prior trainings. 
Depending upon the quality of the match, the monitor displayed a statement 
identifying a "Good recognition" or that the software was "Unable to 
recognize" and the researcher decided what step to take next. 

The next steps options available to the researcher were: 

1. If the training was recognized as adequate, then another sound was 
recorded. 

2. If the training was recognized as adequate, the subject used the sound 
to activate pages displaying the image related to that sound in the STS 
module of the software. 

3. If the training was not adequate, then the entire training sequence for 
that sound can be repeated 

After the third sound had been recorded the reliability of the system to 
recognize and match each of the three sounds to their assigned pictures was 
degraded. The degradation was inconsistent from day to day and therefore 
once sounds were trained in the manner described above, a display page 
(specifically configured for test purposes) was displayed on the computer 

monitor by the researcher. The subject was requested to "tell me about " 

for each of the three objects displayed. If the sounds activated the prompted 
items displayed on the page, then field testing continued in a standard manner; 
if the sounds did not activate correctly, then they would be retrained. 
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Field Testing 

In the middle of August 1990, phase one of the STS/Graphics system was 
introduced to the six subjects. This provided a full screen display of an image. 

Intervention started in July 1990 with all six subjects. During initial sessions, 
one research assistant videotaped while another took notes on activities during 
a session and also worked with the subjects with the researcher. 

The computer system was setup in a room approximately 40 X 20 that was 
isolated from other rooms in the school for training purposes. Furniture was 
moved to provide the least distracting environment and best access to the 
system. Objects to be used for a specific session were placed in a box near 
the researcher or the research assistant. 

The subject entered the room, greetings were then exchanged, and the subject 
was introduced to everyone until she became comfortable with the research 
team. The subject was positioned in front of the system. 

The procedure for preparing and working with each subject varied slightly so it 
will be briefly described for each one. 

Subject TL - At the beginning of a session. Subject TL would be very active, 
examine everything in the room, especially the video camera. Once finished, 
she would sit in front of the monitor. Her chair would need to be pushed close 
to the table to keep her attending and sitting. Sometimes she would wear the 
microphone and on other occasions she would choose to hold it. 

Later in the project, to assist her with attention. Subject TL used a head-set 
instead of the external speaker for the STS/Graphics system. In a few of the 
last sessions, she sat on the researcher's lap because she was very active and 
did not want to remain sitting. 

During sessions with particular research assistants. Subject TL would work for 
about 20 minutes. Each time she successfully activated the system, she would 
"High Five" one assistant and then turn to interact with the. other and any other 
people in the room. 

Subject TL went through the steps of the research design as defined. She had 
problems with producing consistent L sounds that the computer could 
discriminate when "lip gloss" and "lotion" were introduced. She used "li" for 
"lip gloss" and "to" for "lotion" but they were too similar for the computer to 
discriminate. 

Subject MP - Subject MP normally arrived with a "goodie" bag which was 
examined by a research assistant to see which preference items Subject MP 
had brought from home. Items that were to be used as preferred items would 
be but in the box with others and the bag would be put out of sight. Subject 
MP was initially resistant to wearing the microphone, but over an extended 
period of time, she began to wear it most of the time. Occasionally she would 
choose to hold it. 
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She was nervous and apprehensive at times and normally answered "no" to 
questions and initially said "no" to the computer but quite quickly settled down 
and used her sounds. When she decided a session was finished, she went 
back to "no" and when requested to use her sounds, would say "no". 

Subject MP stepped through the research design. 

Subject DL - Subject DL arrived with a bag, containing her school schedule and 
communication book, which she gave to a researcher. She greeted everyone 
and then would be happy to sit in front of the monitor and would get to work. 
She always wanted to scoot her chair forward and would need help. Subject 
DL worked both with and without the light. 

Subject DL worked through the research design as described in the data 
analysis sheets. She advanced to a 2X2 display with a single image. 

Routine 



Once the subject was seated and comfortably positioned the first display was 
loaded on the computer. The researcher would use prompt 1 , if there was no 
response or it was incorrect then the next prompt in the sequence was used, 
etc. Once the system was activated by the correct utterance, the activity or 
object was provided to the subject. If the subject was working in the dark, 
then as soon as the computer reacted to her utterance the light was turned on. 
The activity was pursued for a few moments, e.g., one page of a book read, 
one bead slotted, one bit of food. For X-10 items they timed out at several 
seconds. Later it was found that the research assistants were manually 
controlling these activities so the timing depended on human judgement. 

When the computer was activated, the subject not only received the requested 
object/item but also was praised by the researcher. Often the subjects would 
interact with other members of the team to receive affirmations. In cases 
where a subject showed a reluctance to terminate an activity after the allotted 
time, another research assistant would ask to see the book or activity for a 
while. This strategy seemed to be successful. Once the activity was 
requested again through the computer, the research assistant would give it 
back to the subject thanking them for letting them see/share the activity. In 
the case of books, this process was used but the team member asked to see 
the pictures at the end of each page. 

This process was repeated throughout the session. Sessions were normally 
ended because of time limits. 



Results 

Subject TL, who used direct selection, progressed through training that began 
with selecting a single item on the display and ended with selecting from 
multiple items that were topically similar, e.g., beauty aids, food items. Subject 
2, who used scanning, did not progress past a single image on a 2 X 2 display 
at the time data collection terminated. 
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The two subjects learned to successfully communicate and to make choices via 
the STS/Graphics System. They successfully learned the cause/effect 
relationship between vocalization and action. Subject TL, the subject using the 
direct selection access mode, learned that different vocalizations were 
associated with different functions. Variability existed between the subjects in 
their level of proficiency in operating the system. Subject TL demonstrated an 
understanding of the use of the system with two choices available, both of 
which were highly motivating. Her performance deteriorated upon introduction 
of a third choice. It is unknown if this was due to boredom, dissatisfaction 
with the choices available, inability to differentiate the target vocalizations with 
three choices, or a lack of understanding of the use of the system with three 
choices. Subject 2 demonstrated understanding of use of the scanning access 
mode to make choices. Although she had multiple items from which to 
choose, due to lack of time with the system, she never advanced beyond a 
single item on the display at one time. The graphics component of the system 
was appropriately used by the subjects in this application. 

Aspects of the STS/Graphics System requiring further refinement are (a) 
facilitating the additional training of the system that sometimes becomes 
necessary due to the subjects' improved vocalization over time, (b) improving 
the accuracy of the system's voice recognition in noisy environments, (c) 
increasing the flexibility of the microphone placement, and (d) improving the 
portability of the system. 



Conclusions 

The STS/Graphics System is a viable method of communication and 
environmental control for children and adults having mental retardation and/or 
physical disabilities. The system is effective in teaching persons with mental 
retardation to use their natural speech to communicate and to control their 
environment, and thereby to become more independent. The graphics interface 
provided a symbolic visual aid that served to highlight the options available to 
subjects and facilitate the act of making a selection. Vocalization is a natural 
means by which to empower an individual and proved to be a robust access 
mode for assistive technology with this population. Additional applications of 
the STS/Graphics System for assessment and educational uses will be 
explored. Given the enhancement in personal choice making that the 
STS/Graphics System provides its subjects and its potential to make a 
contribution in other functional areas, future research on the application of 
voice recognition technology to the needs of children and adults with mental 
retardation should be revealing. 

The efficacy of the STS/Graphics System with each subject was evaluated with 
a changing conditions single-subject experimental design (Alberto & Trouknan, 
1990; Smith, 1979). 
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Subject DL 

Subject DL, the student who used scanning as her access mode, achieved the 
criterion level of performance under Condition A, a 1 x 1 matrix (a full-screen 
single-symbol display) using each of eight different symbols, by the ninth 
experimental session. (See Figure 1.) During Session 8, Subject DL was 
exposed to both a 1 X 1 matrix and a 2 X 2 matrix displaying a single symbol. 
Her performance during these nine sessions was highly variable, ranging from a 
low of 0% of correct responses to a high of 100% of correct responses. Her 
mean performance during Condition A was 58%. 

Subject DL's performance under Condition B, a 2 x 2 matrix displaying a single 
symbol (and three blank cells), maintained this high variability through Session 
17. Thereafter, Subject DL's performance showed much greater stability, 
ranging from a low of 47% to a high of 77% of correct responses. Her mean 
Performance during Condition B was 59%. 

The cognitive demands on Subject DL were markedly different under Condition 
A than under Condition B. The task under Condition A required Subject DL to 
vocalize while a single symbol was visible on the display. The vocal response 
could vary within a relatively wide range and still be recognized by the 
STS/Graphics System. Also, the same vocalization was acceptable for any of 
the eight symbols displayed. The only requirements placed on Subject DL were 
that the vocalization exceed the voice-recognition threshold of the system and 
that it be spoken within a reasonable latency after the initiating prompt from 
the experimenter. That is, since in the scanning mode a subject's vocalization 
merely acts as a "voice switch", the exact nature of the vocalization is less 
important than its timing. And in the case of a 1 x 1 display, there is 
essentially no scan (i.e., there is only one symbol centered on the display) and 
consequently the timing demands are also minimal. The "correctness" of the 
response was indicated to Subject DL by the resultant activation of speech 
output or an electrical device. 

On the other hand, the task under Condition B on any given trial displayed a 
symbol in any of the four cells in a 2 x 2 matrix. Subject DL was required to 
recognize that this divergent visual display served the same function as the 
previous, simpler visual display. To lean that the colored "box" that surrounds 
a particular cell and that continually moves from cell to cell is a discriminative 
stimulus for vocalizing; and to anticipate the movement of the box to the cell 
containing the symbol. The critical cue was actually a discriminative-stimulus 
complex composed of the symbol and the box. When these two stimuli were 
combined, they were discriminative for the vocal response. The "correctness" 
of the response, i.e., a recognizable sound that was vocalized while the box 
was surrounding the symbol, was indicated to Subject DL by the resultant 
activation of speech output or an electrical device. 

Despite the increased visual complexity and task demands under Condition B, 
Subject DL's performance did not deteriorate compared to that under Condition 
A. This suggests that Subject DL came to understand the general utility of the 
system as an outlet for communicative intent and to learn the more demanding 
operational requirements of the system. In fact, the increased stability of 
Subject DL's responding across the last 19 experimental sessions suggests that 



Page 23 of Method 



STS/Graphics 



Grant No. H180P90015 



her learning continued to strengthen and her use of the system became more 
reliable as her experience with the system increased. Based on this evidence, 
we have every reason to believe that Subject DL would continue to refine her 
use of the system with a 2 x 2 matrix with a single symbol and transfer quite 
readily this learning to a 2 x 2 matrix with multiple symbols. 

Subject TL 

Subject TL used direct selection as her access mode with the STS/Graphics 
System. Under Condition A-|, a 1 x 1 matrix (a full-screen, single-symbol 
display) using each of three different symbols, her performance increased from 
25% correct responses during the first experimental session to nearly 94% 
correct responses during the second session. (See Figure 2.) 

Under Condition B-|, a 2 x 2 matrix displaying a single symbol (and three blank 
cells), Subject TL averaged 72% correct responses with two of the symbols 
used in Condition A-| . Under Condition A 2 , a return to a 1 x 1 matrix. Subject 
TL achieved a criterion level of performance with all six symbols by the next 
three sessions (Sessions 7-9). During Session 9, Subject TL was exposed to 
both a 1 X 1 matrix and a 2 X 2 matrix displaying a single image. 

Under Condition B 2 , a return to a 2 x 2 matrix displaying a single symbol. 
Subject TL's performance ranged from a low of 27% to a high of 95% correct 
responding. Her mean performance was 67% for the 10 experimental sessions 
(Sessions 10-19). Under Condition C, a 2 x 2 matrix displaying two symbols 
(and two blank cells). Subject TL averaged 77% correct responding across 
three sessions. And finally. Subject TL averaged 64% correct responding 
across three sessions under Condition D, a 2 x 2 matrix displaying three 
symbols (and one blank cell). 

The cognitive demands confronting Subject TL changed systematically access 
conditions. The task under Condition A required Subject TL to learn that the 
particular prompting comment from the experimenter was a discriminative 
stimulus for a specific vocal response. The particular picture symbol on the 
display also served as a discriminative stimulus for that same vocal response. 

In function, the experimenter's prompt and the symbol served as "relevant 
redundant cues" (Engelmann & Camille, 1982). The "correct" vocal response 
was operationally defined by the voice template and recognition threshold of 
the STS/Graphics System, i.e., a correct response was one that was recognized 
by the system. Feedback to Subject TL on the correctness of her response 
was in the form of the resultant activation of the speech output or electrical 
device. 

The task under Condition B required Subject TL to transfer her responding to a 
screen that still displayed only a single symbol at a time, but now the symbols 
were reduced to one-fourth of their original size and appeared in any of four 
cell locations on the screen on any given trial. Subject TL made the transition 
to this new visual interface with little difficulty. 

The task under Condition C presented dramatically different circumstances to 
Subject TL. While the screen continued to be divided into four cells, for the 
first time more than one of the cells contained a pictographic symbol, i.e., two 
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symbols were displayed at the same time. The symbols would appear in any of 
the four cells on any given trial. Now a symbol did not function as a relevant 
redundant cue for correct responding. Simply emitting one of the vocalizations 
associated with either symbol was not adequate. The only cue available to 
Subject TL to indicate what her "correct" response should be was the most 
immediate comment from her communication partner (the experimenter). This 
approximated the conditions under which a natural conversation might occur. 

If Subject TL emitted the vocalization that was appropriate to this comment, 
then correct feedback was provided (in the form of spoken output or device 
activation). The reason for focusing first on the subject's responding in a 
conversation rather than initiating a conversation was logistical: the 
researchers wanted to be certain the subject was not simply emitting, or 
cycling through, all of the vocalizations that were represented on the display, 
regardless of their appropriateness or meaning. 

Subject TL's performance was not disrupted by these more complex task 
demands. She had to learn more precise discriminations, i.e., while the two 
symbols that were displayed served to reduce the range of potential responses 
from six vocalizations to two, to determine which of the remaining two options 
she had to attend solely to the comments of the experimenter. Subject TL 
learned this quite readily. 

The task under Condition D presented to the subject more available options for 
responding. Three symbols were now displayed in various cell locations on any 
given trial. The subject had to learn that the discriminative stimulus under 
these conditions was still the communication partner's most immediate 
comment. During the initial session of this condition (Session 23), Subject TL's 
performance was disrupted relative to Condition C. She responded correctly 
only 36% of the time during Session 23. However, she picked up her 
performance to 94% correct responses on the very next session. During the 
last three sessions of this research, Subject TL averaged 63% correct 
responses under Condition D. These data suggest that she not only 
understood the utility of the STS/Graphics System as a vehicle by which she 
could communicate, but also generalized this understanding to a wide and 
unpredictable array of visual interfaces within a 2 x 2 matrix. 

Anecdotal data from the experimental sessions provide intriguing suggestive 
evidence to account for the variability in Subject TL's performance from session 
to session across conditions. Experimenters repeatedly reported that, as her 
experience with the STS/Graphics System progressed, she increasingly emitted 
the vocalizations for her most highly-preferred spoken-outputs/activations, a 
request for cheese-and-crackers and a comment about nail polish — even on 
those trials in which the symbols for these two items were not displayed. In 
other words, Subject TL's understanding of the purpose of the system and her 
actual use of the system advanced from that of responding to a partner's 
questions/directions to that of initiating an interaction. Even though these 
vocalizations were technically categorized as "incorrect", because they did not 
respond to the experimenter's questions/directions, the experimenters 
expressed a high degree of confidence that they were based not on Subject 
TL's inability to understand the question/direction nor her inability to come up 
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with the "correct" vocalization. Rather, they were a function both of her more 
complete understanding of the system as the means by which she could 
express herself in general and her increasing assertiveness in controlling her 
environment. 

These observations suggest a remarkable change in a subject with severe 
mental retardation. Not only was she generalizing her use of the system far 
beyond what she had directly been taught, but she also was no longer 
dependent on the concrete visual referents for her vocalizations. In other 
words, in typical system use, the symbols function as a type of cognitive 
prosthetic aid that "remind" the subject of the set of "eligible" vocalizations 
from which she should choose to emit one. Subject TL progressed past the 
point of needing this aid and was selecting her vocalization, solely from 
memory to communicate her intentions. This situation more closely 
approximated the "give and take" of a natural conversation in which some of 
the communications are responses to the communication partner, some are 
initiations. The discriminative stimuli for the particular initiations often include 
internal stimuli, i.e., internal need states. The researchers are quite pleased to 
have observed this progression in the subject who was a direct selector. The 
dynamics of this progression and its relationship to more powerful system-use 
by subjects with cognitive disabilities will be the focus of future research 
efforts. 

In conclusion, both Subject DL who accessed the STS/Graphics System via 
scanning and a vocal response and Subject TL who accessed the system via 
direct selection using vocal responses learned how to use the system 
appropriately to communicate. They progressed from full-screen single-image 
displays to 2 x 2 matrices that lay the foundation for greatly expanded and 
more versatile communication in the future. The reduced cognitive load on 
system use provided by the graphics visual interface and the reduced physical 
load provided by the voice interface greatly facilitated their communicative 
competence with the system. The ease in incorporating personally-meaningful 
pictographic referents and in configuring screen displays with the STS/Graphics 
System appear to be major factors in accomplishing these advances for children 
with severe cognitive and physical disabilities. 
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Figure 1 . The percentage of trials in which Subject DL responded correctly via 
the STS/Graphics System as a function of daily sessions. 
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Figure 2. The percentage of trials in which Subject TL responded correctly via 
the STS/Graphics System as a function of daily sessions. 
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Sound-to-Speech/Graphics 
Initial Subject Profiles 



Subject # 
& Sex 


Date of Birth 


School 


School 

District 


Disability 


1 - M 


7/11/74 


Travis Jr High 


Irving 


Severe MR, legally Blind 


2 - F 


9/23/81 


A. S. Johnston 


Irving 


MR (accessed 18-23 months) 


3 - M 


3/2/77 


A. S. Johnston 


Irving 


MR, Autistic 


4 - M 


6/9/84 


A. S Johnston 


Irving 


CP, MR, hyrocephalic 


5 - M 


3/24/84 


A. S. Johnston 


Irving 


MR, (difficulty assessing - 18 months?) 


6 - F 


2/24/79 


A. S Johnston 


Irving 


MR, seizure disorder 


7 - F 


4/13/70 


Irving High 


Irving 


Unresponsive to accessment (speech 2 months - Binet) 


8 - F 


5/18/81 


A. S Johnston 


Irving 


CP, hyrocephalic (shunted) 


9 - F 


12/28/79 


A. S Johnston 


Irving 


MR, mild CP (20-30 months - scattered) 


10 -F 


4/11/74 


Irving High 


Irving 


MR, MH, autistic, neurological problems, 
(no assessment) 


M 


9/21/82 


A. S Johnston 


Irving 


(Assessed 2 years 10 months) 


12 - M 


4/1/78 


A. S Johnston 


Irving 


MR, (Assessed 4-6 years) 


13 - F 


7/19/83 


Meadow Creek 


HEB 


Static encephalapathy with spastic displasia 


14 - F 


11/7/69 


Career Center 


HEB 


Severe MR, SH, CP 


15 - F 


1/14/69 


Career Center 


HEB 


OH, MR, CP, SH (2-3 year equivelent) 


16 -M 


10/18/84 


Bell Manor 


HEB 


OHI, SH, Neuromuscular Desease 


17 - M 


10/10/78 


Career Center 


HEB 


Autistic, ISH, Leukemia 


18 - F 


4/5/83 


Career Center 


HEB 


MR, OHI, SH 


19 - F 


6/8/86 


Bell Manor 


HEB 


MR, SH, CP 


20 -M 


2/12/83 


Career Center 


HEB 


Down Syndrome, (speech - exp. 
4-16mths Rec. 12 mths) 


21 - M 


9/17/71 


Career Center 


HEB 


MR, OH, SH, CP (no test results) 


22 - F 


6/10/86 


Donna Park 


HEB 


MR, CP, ISH (getting glasses) 


£.M 


5/26/83 


Meadow Creek 


HEB 


OH, SH, CP 


24 - M 


1 1/30/69 


Career Center 


HEB 


MR, OH, SH, CP (no assessment) 
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HARDWARE AND SOFTWARE 



The Research and Logistics of Producing the STS/Graphics Package 



Hardware Components 

Computer: 286 or 386 Central Processing Unit (CPU), 30 MB Hard Disk, 
Floppy Disk Drive, 640 KB RAM 

Super VGA Monitor 

Paradise Graphics Board 

Microtek Color/Grey Scanner 

Votan 2000 Voice Recognition Board, Microphone & Speaker 
X-10 Controller 
Universal Remote Control 



The Computer System 



The STS/Graphics system software was developed and operated on either a 
286 or 386 CPU. The software has not been tested on a 486 CPU although 
there are no design constraints to prevent this. Therefore, it is expected that 
the software would operate within the 486 environment. The software code 
was developed on three different models of computers: an IBM Computer 
System with a 286 CPU; a Clone Computer System with a 286 CPU; and a 
Clone Computer System with a 386 CPU. 

Hard Disk Drive for Information Storage and Retrieval 

When the computer equipment was purchased for this project, two complete 
systems were acquired. One computer had a 20 MB hard disk drive and the 
other had a 30 MB drive. These drives allowed storage of programs, 
graphics/image files (each file contained the scanned image of one photograph), 
and data for several students at the same time. Image files varied in size 
depending upon the size of the image as it appeared on the monitor and the 
variety of colors in the image. Due to disk space problems, these files were 
stored on floppy disk rather than on the hard disk. 
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Image Capturing Device Selection and Its Use to Create Graphic Libraries 

For this project, color was an essential part of any image that became a graphic 
representation in the system. Each image was captured into the computer by 
digitizing it. Three methods of digitizing images were considered: a scanner, a 
video camera, or a still camera. Use of any of the three technologies would 
create a color graphic image of approximately equal quality. The cost, 
excessive number of components to each system, and complexity of operation 
for both the video and still camera eliminated them from consideration. 

Several scanners were considered, but since color scanning was a requirement, 
most scanners were beyond the designated hardware budget. After 
researching all possible devices, a Microtek Color/Grey Scanner, LS-Series, 
model LS-2400 was selected. LeColor, a custom software program, was 
acquired to interface the scanner and computer since the software which came 
with the scanner was not able to operate within the requirements for the 
STS/Graphics project. 

Images were digitized and scanned from photographs taken of objects familiar 
and preferred to project participants. If the image was reduced in size after 
being scanned, image clarity was reduced and distortion resulted. 

Consequently, prior to the images being scanned, the display size (full screen, 
2X2, 3X3, 4X4 or 5X5) was defined and the image was scanned specifically 
for that size to help eliminate image degradation. Once the image had been 
digitized, PC Paint software (a drawing package) was used to optimize the 
image position and to minimize background. Images were stored as separate 
files on diskettes for future use. 

Graphics Monitor for Color Presentation of Stimuli Items 

At the outset of the STS/Graphics project, the standard for most high 
resolution color monitors was a 640 X 480 pixels display (EGA). To provide 
extremely high quality image displays for the STS/Graphics project, it was 
necessary to integrate a Super VGA monitor and a graphics adaptor card which 
had additional memory which would give a video display of 800 X 1000 pixels. 
Both of these hardware items were new to the market and had little support 
software. After extensive research, The PCX Programmer's Toolkit, of Genus 
Microprogramming, was identified as providing both the correct video display 
for 800 X 1080 pixels and the software drivers. This allowed the memory on 
the video adaptor card to be used to display many high quality images 
simultaneously on the monitor. The Paradise Graphics chip set was an 
accepted graphics standard when the STS/Graphics software was developed, 
so this the graphics software was developed to operate in that mode. 



The Votan 2000 Voice Recognition Input/ouput Board Serving the Voice 
Recognition and Speech Output Function 

The Votan 2000 Voice Recognition Input/output board, which is housed 
internally in the computer, had been used by the Bioengineering Program in a 
previous speech recognition project. At the beginning of this project, new 
voice recognition technology was researched and it was confirmed that the 
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Votan 2000 was still one of the most reliable voice recognition boards- 
available. Based on this finding, the existing boards were used in this project. 

Votan was in the process of developing software libraries for operation with 
their series of voice boards. The Bioengineering Program waited for the 
completion of these libraries to integrate them with the STS/Graphics software. 
When the libraries were delivered, it was found that they had been developed 
for only one C compiler and that they could not operate in the development 
environment being used by the programming staff. Several weeks were spent 
working with Votan to evaluate whether they could modify their source code to 
operate with the various compilers. It was concluded that it would be several 
months before Votan could possibly make the necessary changes and that it 
was not a priority activity for them. Once this was known, the Bioengineering 
programming team rewrote software code from the Pascal language which was 
the language used for the previous Bioengineering Program voice recognition 
project. During this process, the software code was converted to "C". The 
software code eventually operated both the voice recognition and voice record 
functions needed for this project. 

Microphones and Speakers for the Voice Input/Output to the Votan Board 

The Votan 2000 Board is supplied with a microphone and speaker unit. The 
speaker measures approximately 2X3X4 inches, and the microphone is 
mounted on a gooseneck on the top of the speaker. Both components are 
attached by audio cables to the Votan board which is housed in the computer. 
During the evaluation of the subjects, it was found that in some cases the 
microphone/speaker was a distraction to the subject. Unsuccessful attempts 
were made to position this unit in a less obtrusive location. While searching for 
an alternative to this situation, a variety of microphones were tested for use 
with the Votan 2000 Board. These included a conference microphone 
manufactured by Realistic, which appeared to be a flat sheet of metal 
approximately 4X5 inches. Even though the subjects were unaware of this 
unit, this microphone was unreliable when interfaced with the Votan Board. A 
second Realistic microphone, which was designed for use in presentations, was 
tested. This unit was very small and clipped onto the users clothing on a collar 
or somewhere that was close to the users' mouth. Performance for this 
microphone with the Votan 2000 was acceptable and students were less aware 
of it than they had been with the Votan microphone. In some cases, subjects 
preferred a researcher to hold the microphone which was not a distraction. 
Additional microphones were tested, but none were able to maintain high 
enough quality of speech to interface satisfactorily with the Votan 2000 Board. 

When a microphone other than the Votan was used, a Realistic speaker was 
interfaced for speech output. This unit provided greater volume than the Votan 
unit and in time it replaced the Votan speaker even when the Votan 
microphone was used. 

X-10 and Universal Remote Control for Environmental Control Function 



X-10 controllers, which are electrical modules for environmental control, are 
supplied with a serial interface and software drivers for computer operation. 
Bioengineering programmers accessed the X-10 source code and converted it 
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to the "C" programming language. This code operated reliably on an IBM 
PC/XT computer, but when it was transferred to a faster CPU, reliability 
problems were experienced. The development team worked extensively with 
X-10 technicians to resolve this problem. The first identified problem was that 
the speed at which the signal was sent from the computer to the X-10 
controller was too fast. This was resolved by slowing down the signal. 

Further problems were experienced during the software development phase 
when code to operate the X-10 Controller Unit was integrated with other 
modules of the STS/GRAPHICS software. The X-10 operated unreliably. 
Problems ranged from X-10 failure to complete serial port failure. The difficulty 
was overcome by using Greenleaf Software Library routines for port control. 
Because the problem seemed corrected, it was assumed that the problem had 
been solved. It was found later that the "solution" had only solved the obvious 
problem and not the underlying cause. 

The underlying cause for these intermittent failures were traced to hardware 
difficulties. After having three serial ports replaced on various computers, it 
was found that there was an incompatibility between the Votan 2000 voice 
input/ouput board, utilization of the computers serial port, and the computer 
chip set. If the Votan board was operated while the X-10 signal was passed 
through the serial port, then X-10 failure was experienced. The types of 
failures depended upon the X-10 instruction that was sent, but all failures were 
irreversible. The problem was solved after extended trouble shooting by only 
operating the software on a computer system that had a chip set other than 
"Chip" brand chips. The ultimate result of this finding was that computers built 
with this chip set had to be avoided for operation of this software. 

The X-10 controller can only operate appliances through house wiring. Thus it 
was necessary to integrate a method to control devices which operate through 
infrared control where house wiring is not used. The One-for-AII Universal 
Remote Control unit was the only infrared controller available with an RS232 
interface at the time of this project, so was selected to control all infrared 
operated devices. The One-for-AII was sold with software to interface with the 
RS232 port. Unfortunately, after the product was shipped, it was found that 
the software was not functional. Universal Electronics, who developed the 
One-for-AII, lost the source code for their software and were unable to assist 
us in making the RS232 port functional. After researching the problems, 
operation of the unit was accomplished using original source code developed 
by Bioengineering programmers which was integrated with Greenleaf libraries. 

Single Switch Control for Alternate Computer Access 

Single Switch Control was accomplished using a PC-Pedal switch which was 
manufactured and sold by Brown and Company, Inc. This was the least 
expensive single switch that operated on IBM computer using the RS232 port. 



STS/Graphics Development Software 

Software development was done using Borland's Software Development 
Package which includes the Turbo C compiler, the programmers tool kit, and 
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the debugger. It was found that in most instances, this development 
environment was compatible with other resource libraries. 



STS/Graphics Custom Software Development 

The STS/Graphics system operating software was designed to be modular. By 
developing the software in a modular format, it allowed for the independent 
development of the various sections of code. Each section of code was 
written, tested, debugged and then integrated into the total software package. 
Major modules were for voice input/output, scanning, graphics displays, 
environmental control, data collection and retrieval, report generation, and 
STS/Graphics operation. 

The scanning and graphics modules were developed simultaneously followed by 
the voice input/output module, environmental control, and STS/Graphics 
operation. The final module which was developed was data collection and 
reports. These modules were dependent upon all other modules for 
completion. 



Impact on the Research Project 

Delays in completion of the software were due to ongoing hardware problems 
and support software failures (see hardware section). Consequently, initial 
evaluation and intervention with the subjects were initiated with the scanning 
and graphics module and a intermediate version of the voice input/output 
module. The voice training portion of the software which display full screen 
images of objects was added to the project in August, 1990. 
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Figure 1. Sound-to-Speech/Graphics Hardware Configuration 
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Description of the Software and 
Hardware Development and Design 



The research described herein involves the design and development of a voice- 
operated AAC computer system that was used by children and young adults 
with mental retardation to communicate and to control their environment 
through no greater effort than the vocalization of simple sounds. 

The STS/Graphics System is designed to research the potential of voice 
recognition technology for persons with mental retardation and is an extension 
of previous research and development conducted by Brown and Cavalier and 
their colleagues. In earlier research, a voice-activated computer system was 
developed and evaluated with a woman diagnosed with mental retardation and 
severe multiple disabilities, who needed pervasive support. The woman learned 
to voice activate the system to control electrical devices in her immediate 
environment. 1,2 This was followed by research on a multi-user system with 
environmental control and speech output capabilities. Neither of these studies 
employed a picture-based display to assist the user in making choices. 

The next generation in this research track, the STS/Graphics System, 
incorporates a graphics display. Key features of the STS/Graphics System are 
as follows: (a) photographic-quality computer-generated images of 
items/persons in the user's natural environment that are presented on a display 
to represent choices among spoken communications and environmental control 
activations; (b) vocalizations, whether intelligible or not, that are used to select 
choices from the image display; (c) physical switch activations and traditional 
keyboard inputs that are alternate access modes; (d) voice and keyboard 
access modes that can be used to make choices via direct selection; (e) voice 
and switch input that can be used with linear scanning and row/column 
scanning; (f) speech output that is in the form of digitized, easily-understood 
speech and that is age and sex appropriate; and (g) data on system use that is 
automatically recorded, thereby facilitating the development of appropriate 
training strategies. Figure 1. Sound-to-Speech/Graphics Hardware 
Configuration depicts the hardware components of the STS/Graphics System. 

A more detailed description of the STS/Graphics System follows: 

Hardware 

The following list of hardware components represents the minimum 
configuration required for the operation of the STS/Graphics System. Enhanced 
performance can be obtained with a 386 or 486 microprocessor and larger hard 
disk capacity. Additional Random Access Memory (RAM) will not enhance 
performance. 
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Hardware Components 

o International Business Machine (IBM) Personal Computer (PC) 
Advanced Technology (AT) or compatible with a 286 central 
processing unit (cpu) 

o 30 Megabyte (MB) of hard disk space 
o Super Vector Graphics Adaptor (VGA) monitor 
o Paradise Professional Video Adaptor 
o Votan 2000 speech recognition board 
o Image capturing device with color capability 
o X-10 Powerhouse and various wall modules 
o 640 Kilobyte (KB) of RAM 
o Floppy drive 

o Microsoft-Disk Operating System, version 3.30 (MS-DOS) or 



Voice Input 

The voice recognition system incorporates the Votan 2000 voice recognition 
circuit board manufactured by Votan Inc. of Fremont, CA and software routines 
unique to the system developed at The Arc. When operating the STS/Graphics 
system, the user's voice functions as a switch and the user "trains" the 
computer by speaking the target vocalization into the microphone. This sound 
is recorded digitally on the hard disk. The training is repeated three times and 
each vocalization is recorded separately. Once the three trainings are stored in 
a user-specific voice template, the STS/Graphics software analyzes the three 
recordings and averages them. For scanning access to the system (for more 
information on scanning, see discussion in Access Mode section), the user 
specific template is comprised of trainings for one unique utterance. For direct 
selection access to the system (for more information on direct selection, see 
discussion in Access Mode section), the user specific template is comprised of 
multiple unique vocalization trainings. Once all of the trainings are stored, the 
system checks to verify that the user's target vocalization will successfully 
match one of the sounds stored in the voice template. If the match is not 
similar enough to the original trainings, then the user must retrain the voice 
template for that particular utterance. 

When the user is actually operating the system and produces a vocalization, 
the system checks to see if the vocalization achieves a "match" with the 
trained vocalizations stored in the voice template. In the event that there is a 
match, the system will acknowledge the vocalization by activating the 
appropriate speech output message or environmental control function; in the 
event of the vocalization not matching any of the trainings closely enough, the 
system will not respond. If the system does not respond, two optional 
corrective actions can be taken. First, the accuracy used by the system to 
match a vocalization to the stored voice trainings can be made less sensitive so 
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that the system is less demanding in its matching requirements. Second, the 
user may choose to repeat the training process to refine a vocalization stored in 
the template for better recognition accuracy. 

Sound-to-Speech Translation and Expansion 

For communication purposes, each user can access speech output messages 
stored in digitized files on the hard disk. Each message can consist of speech 
of up to 8 seconds in duration, thus permitting single words or complete 
sentences to be output. Someone whose voice is age and sex appropriate for 
the user records these messages into the system. The Votan board manages 
this process. 

The system uses an algorithm to compare a user's vocalization with those 
previously stored in a voice template. If a match is found, the input is 
identified and linked to one of the speech output messages. In a sense, the 
user's often brief and unintelligible vocalization translates into intelligible 
speech and expands into a phrase or complete sentence. For example, if a 
user's vocalization for "water" approximates "wuh", the system can 
immediately output "Could I have a drink of water, please." Message files for 
many different users can be stored and accessed via one computer system. 

The number of available files is dependent only upon the size of the system's 
hard disk. 

Environmental Control 

The system incorporates X-10 Powerhouse environmental control technology 
manufactured by X-10 Inc. of Northvale, NJ. The computer sends digital 
encoded signals over existing electrical wiring in the room or building in search 
of a lamp or wall "module" with the same identification code. The modules 
plug into wall outlets. The target module responds to the control signals 
transmitted from the computer and activates or deactivates the appliance 
plugged into it. Any voice (or other) input can be linked to the activation and 
deactivation of designated electrical devices. A single voice input can also be 
linked to any combination of speech outputs and device activations, e.g., the 
vocalization "pa" translates into "I'm going to watch TV now", and the 
television set turns on. 

The system also uses a One-for-AII universal controller manufactured by 
Universal Electronics, Inc. of Signal Hill, CA, to control any audio/visual device 
that operates through the use of an infrared remote controller. The universal 
remote controller operates via a serial I/O port located inside its battery 
compartment. Using a simple communications protocol, the computer system, 
through an RS-232 port, electronically "presses the buttons" of the remote 
control unit to operate the device. 

Graphics Interface 

The system displays on the video monitor photographic-quality images of 
objects, appliances, and people in the user's environment. The size of the 
images, as well as the number of'images appearing on the display, are 
customized to the user. 



Page 9 of Software/Hardware 



STS/Graphics 



Grant No. H180P90015 




Figure 2. Overview of the Main Menu 
for Sound-to-Speech/Graphics Software 
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This is achieved by digitally scanning photographs of items into the computer 
using an image capturing device (a scanner or video camera) and then sizing 
the image using image manipulation software such as PC Paintbrush. Extensive 
image libraries are created for the users. An image library consists of disks 
containing scanned images of items that have personal meaning to the user. 

The library is arranged by category and serves as an organizational model for 
the teacher or trainer. In this way, the same images are used by a variety of 
subjects in their customized displays. 

Displays range from a single image to a 5 x 5 matrix of images. On a 13 inch 
monitor, the size of each of the four images in a 2 x 2 matrix is approximately 
3.75 X 5 inches; the size of each of the 25 images in a 5 x 5 matrix is 
approximately 1.5 X 2 inches. The graphics display can also "flip" to a second 
page of choices for each user. The two pages of images permit the 
configuration of a hierarchical system of choices for users who understand the 
concept of categorical clustering. In such a system, the images displayed on 
page one represent superordinate categories, e.g., an image for the "food" 
category, an image for the "clothing" category, an image for the "sports" 
category, etc. The selection of a page-one image displays a second page of 
images all related to the particular category selected on page one. However, it 
is not mandatory that images on page-one represent super-ordinate categories 
for the second page. On the second page, one cell is always designated as a 
"return” to page one. 

The graphics images in this system act as cognitive prosthetic aids that serve 
to remind the user of the speech outputs and device activations that are 
available. While a user's voice input (or switch or keyboard input) initiates the 
sequence of electronic events that results in some functional output, the 
graphics images provide a symbolic representation of output choices that are 
"live", or available, at any one time. 

Access Modes 

The STS/Graphics software provides two methods of accessing the system: 
direct selection and scanning. In the direct selection mode, a user vocalizes a 
different sound for each image that is presented on the display. The scanning 
selection mode is operated in two ways: linear scanning or row/column 
scanning. In either scan mode, the system requires only a single vocalization 
to access all choices and the vocalization does not have to be consistently 
pronounced from one occurrence to the next. The vocalization serves to stop 
the scanning sequence. Once the scan halts, the functions execute that are 
associated with the image on which the scan stopped (that is, speech output, 
environmental control, or both). Alternately, the scanning can be stopped by 
using a single switch. 

Software Design 

The customized system software is made up of four major components: 

Speech Manager, Setup, Sound-to-Speech, and Reports as shown in Figure 2. 
Overview of the Main Menu for Sound-to-Speech/Graphics Software. 
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1. Select an audio message category. 

2. Add an audio message category. 

3. Delete an audio message category. 

4. Modify an audio message category. 

5. Exit. 



1. Record an audio message. 

2. Modify the text description of a message. 

3. Print a list of the messages. 

4. Delete an audio message. 

5. Modify an audio message recording. 

6. Exit. 











1 . and 5. The teacher 
is prompted with a 
beep to record the 
message. 




2. The teacher is 
given on-screen 
prompts to type 
(or edit) the 
description of 
the selected 
message. 




3. The list of 
categorized 
messages and 
their descriptions 
is printed. 




4. The teacher 
selects the 
message to be 
deleted from the 
displayed list and 
activates the delete 
function. 


6. Exit. | 



Figure 3. Overview of the Speech 
Manager: Recognition Function for the 
Sound-to-Speech/ Graphics Software 



(Start 



t. Tram a vocalization. 

2. Enter/eoit Denunciation text. 

3. Delete a training. 

4. Help. 

5. Exit. 




Figure 4. Overview of the Speech 
Manager: Record Function for the 
Sound- to-Speech/Graphics So ft ware 



me 



Page 12 of Software/Hardware 



STS/Graphics 



Grant No. H180P90015 



Speech Manager 

The Speech Manager has two discrete functions: the Record Function and the 
Recognition Function: 

Recognition Function, as described in Figure 3. Overview of the Speech 
Manager: Recognition Function for the Sound-to-Speech/Graphics Software, 
creates and manages the voice template that contains the user's recordings of 
single or multiple vocalizations. In the direct selection mode, a different 
vocalization is linked to each image on the display. In the scanning access 
mode, only one vocalization is stored in the system and serves to halt the scan 
sequence on the desired image, resulting in a selection. 

The Record Function, as described in Figure 4. Overview of the Speech 
Manager: Record Function for the Sound-to-Speech/Graphics Software, 
creates the speech output message files and groups these individual audio 
messages into categories appropriate for the user or setting. New categories 
and messages can quickly and easily be added or deleted. Each audio message 
has a corresponding text description in its storage file. 



Setup 

The Setup component of the software, as described in Figure 5. Overview of 
the Setup Function for the Sound-to-Speech Graphics Software (illustrated on 
the following page), is used to combine image files, audio output messages, 
environmental control, and infrared commands and to assign them to specific 
cells in a matrix (from 1 X 1 to 5 X 5). When all of the desired cells in a matrix 
are filled with images (it is not mandatory to put an image in all cells), then the 
information is stored as a "page" file. Each page contains different graphics 
choices and the user can "flip" between two pages as explained in the 
Graphics Interface section. 



Sound-to-Speech 

Once the system has been configured for use, the Sound-to-Speech component 
presents the pages to the user as described in Figure 6. Overview for the 
Sound-to-Speech Function of the Sound-to-Speech/Graphics Software 
(illustrated on the following page). It is the interactive portion of the system 
and the only one with which a user interacts for communication and 
environmental control. Data collection on user activity occurs whenever 
Sound-to-Speech is active. The collected data can be output as a report. 
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Start 



1 . Add a new display. 

2. Modify a display. 

3. Exit. 



The teacher chooses the 
dimensions of the graphic display: 
1x1. 2x2. 3x3. 4x4. or 5x5. 



The teacher chooses the access method for the 
display that will be configured: linear scanning, 
row— column scanning, or direct selection. 



The teacher is presented with a pull-down 
menu of actions that can occur when a user 
chooses a cell, and a pull-down menu of 
images available for the selected user. 

Actions include playback of an audio recording, 
X-1 0 environmental control, universal remote 
(infrared) control of electronics, or any 
combination of playback and X-10 or infrared. 



The teacher is presented a miniature 
version of the matrix. The teacher positions 
the cursor inside the cell (row and column) 
to which an image and an action will be 
assigned. 



Once the cursor is positioned, the teacher 
selects an image and an action from the 
pull-down menu and assigns them to the 
selected cell. 



The teacher can press <ESC> at any time 
to leave the program, with a prompt to save 
the configured display. 




When the user selects an image by vocalizing, 
the corresponding action occurs (i.e. playback 
of an audio recording; X-10 environmental control; 
universal remote control of television. VCR, or CD 
player: or any combination of playback and X-10 
or infrared). 



An entry is written to the log file, indicating the action 
that has occurred along with a time/date stamp. 



The program waits for the teacher to press the 
space bar to continue. 



The teacher can press <ESC> at any time to leave 
the program. 



Figure 6. Overview for the Sound- 
to-Speech Function of the Sound- 
to-Speech/Graphics Software 



Figure 5. Overview of the Setup 
Function for the Sound-to-Speech 
Graphics Software 
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Reports 

The system generates activity reports regarding the operation of the system. 

As each user's choice is activated, a record of the interaction is written to a 
data file. This consists of the time of day, the number of the cell selected on 
the matrix display, the name of the image displayed in that cell a that time, and 
the text of the speech output or the name of the environmental device that is 
activated. The report also includes demographic information on the user, the 
access mode, the name of the particular matrix display, the dimensions of the 
matrix, the sensitivity setting on the voice recognition board, the matrix and 
scanning box (if one is used), colors, the names of the various images and their 
cell locations, their associated speech outputs or device activations, and 
phonetically-spelled approximations of the user's voice inputs for the images. 
When the teacher or user so desires, the software can aggregate and analyze 
these data on the user's activities and can generate on-screen and/or hard-copy 
print-outs. 
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The Sound-to-Speech Software 




communication and environmental control system using a graphics interface 




HARDWARE REQUIREMENTS 



PC/AT Microcomputer with 640 Kbytes of memory and a 30 
Mbyte hard disk drive 

A VCA color monitor 

A color scanner 

A mouse 








SYSTEM FEATURES: 



Produces digitized speech 
Activates/deactivates small appliances and toys 
Uses selections made from graphics screen display 
Selections activated by users vocalization/s 



Vocalization does not have to be recognizable speech 





ADDITIONAL SYSTEM FEATURES: 



Produces photographic quality graphic images 
Images can be rapidly scanned into the system 
Customized graphics screens for users 
Multiple access modes: 

Scanning (row/column, continuous linear, direct) 
Direct Selection (requires range of vocalizations) 




GRAPHICS CONFIGURA TION 



Variable matrix size 



Multiple access modes 



Picture location in matrix customized to user 



Activity associated with each picture 

{e.g. picture of light; turns on the light and produces speech 
output "I want the light on") 











GRAPHICS D/SPLA Y 



Allows a matrix of 4 to 25 pictures 
(2 x 2, 3 x 3, 4 x 4, or 5 x 5) 

Up to 4 levels of screens available 
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KEY CHARACTERISTICS OF USERS 



Unintelligible speech 



Limited cognition 



Severe physical limitation 



Unable to use standard mechanical adaptive switches 
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/mimmmmmmmmmmmmmmmmmmttmmmmm/ 

/* SCRIO - Routines which directly access the video screen 
/* 

/* 

/* 

/mmmmmimmmmitmmmmmmmmmmmmmmm/ 

/* Modification Log 

/tmmmtimmmiimmttmmmmmimmmmmtmtmtim/ 

/* Version Date Programmer Description 

— */ 

/* 

/* V01.00 112787 Bob Withers Program intially complete. 

/* 

/* 

/mmmtmmmmmmmtmtmmmmmmttmimmmmm/ 



*/ 

*/ 

*/ 

*/ 

*/ 



*/ 

*/ 

*/ 

*/ 



#include <stdlib.h> 

♦include <stddef.h> 

♦include <dos.h> 

♦include <string.h> 

♦include "win.h" 

♦define MAXDIM ( array ) (sizeof (array) / sizeof ( array [0 ]) ) 



♦def ine 
♦def ine 
♦def ine 
♦def ine 



SCR_BYTES_PER_ROW 
CGA_MODE_SEL 
CGA_ENABLE 
CGA DISABLE 



160 

0x3d8 

0x29 

0x21 



♦define BIOS_VID_INT 0x10 

♦define BIOS_VID_SET_CRTMODE 0 

♦define BIOS_VID_SET_CURSORTYPE 1 
♦define BIOS_VID_SET_CURSORPOS , 2 

♦ define BIOS_VID_GET_CURSORPOS •' 3 

♦define BIOS_VID_SCROLL_UP 6 

♦define BIOS_VID_SCROLL_DOWN 7 

♦define BIOS_VID_WRITE_CHATTR 9 

♦define BIOS VID_GET_CRTMODE 15 



struct sBoxType 
{ 



BYTE 


cUpperLef t ; 


BYTE 


cLowerLeft ; 


BYTE 


cUpperRight ; 


BYTE 


cLowerRight ; 


BYTE 


cLef t ; 


BYTE 


cRight ; 


BYTE 


cTop; 


BYTE 


cBottom ; 



typedef struct sBoxType BOXTYPE ; 



unsigned 
| unsigned 



uScrSeg 

uCsrType 



68 

. = 0xb800 ; 
= 0x0107; 
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short nCurrAct i vePage = 0; 

short nVideoCard = VIDEO_CGA: 

short nScrCols = 80; 

/tttttttttttitittititttttttmtimttmitmmtmtmtitmtmtiMttmt/ 

/* ScrCvtAttr - Test for a monochrome video card and convert the 



/* requested attribute to black & white as best we can t / 

/* and still honor the callers request. t/ 

/ * Parms : $ j 

/ * nAttr - The passed color attribute. t / 

/% * / 

/* Return Value: A converted black & white attribute if the current */ 

/* video mode is monochrome. t / 

/iititutttittiiiiitiittttiititttiiuttittiittttttiiiiiitttimntmmim/ 



static short near pascal ScrCvtAttr ( nAttr ) 
register short nAttr; 

{ 

short nRev, nClr, nBlink, nlntensity; 

if (VIDEO_MONO != nVideoCard) 
return ( nAttr ) ; 
nlntensity = nAttr & 0x40; 

nBlink = nAttr & 0x80; 

nRev = nAttr & 0x70; 

nClr = nAttr & 0x07 ; 

if (REV_BLACK == nRev) 
nClr = WHITE; 
else 
{ 

nRev =. REV_WHITE; 
nClr = BLACK; 

} ! 

return { nRev j nClr ! nBlink j nlntensity); 

} 



/mmmmmtmmtimmmnmmttmtmtmimmimmmmt/ 

/* ScrBnableVideoCGA - Test the current video equipuent for a snowy CGA */ 



/* 

/* 

/* 

/* 

/* 

/* 

/* 

/* 

/* 

/* 

/* 



Parms : 
nStatus 



Return Value 



card. If running on a CGA enable/disable the 
video signal based on the passed parameter which 
MUST be one of the predefined constants CGA_ENABLE 
or CGA_DISABLE. If the current video equipment is 
not a CGA, the routine returns without taking any 
action. 

- Enable or disable the CGA video signal 
CGA_ENABLE or CGA_DISABLE 

None 



/mmmmmmmummmmmmmmmmmmmmummu/ 



RJC 



static void pascal ScrEnableVideoCGA { nStatus ) 
short nStatus; 

{ f 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 
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} 



if (VIDEO_CGA == nVideoCard) 
outp ( CGA_MODE_SEL , nStatus); 
return ; 



/mtimtmmmtmmmmmmtmmmmmmmmtmmmm/ 

/* ScrGetBectSize - This routine will calculate and return the nuaber of t/ 



/* 




bytes required to store a 


/* 




columns by nHeight rows. 


/* 


Parms : 




/* 


nWidth 


- Column width of the screen 


/* 

/* 


nHeight 


- Number of rows in 



It Return Value: Size in bytes required to store the screen rectangle */ 

/iimtmitmmtmmtitttmtiimtmttttiimmitmimmitmittt/ 



screen image which 



rectangle 

the screen rectang 



1 



is 



e 



nWidth* / 
*/ 
*/ 
*/ 
*/ 
*/ 



short pascal ScrGetRectS ize ( nWidth , nHeight) 
short nWidth, nlleight; 

{ 

return ( nWidth * nHeight * 2); 

} 



/muummmmmtmtmmmmmmmmmmmmtmmmu/ 

/* ScrClearRect - This routine will clear a screen rectangle to the 
/* color attribute passed. 

/* Parms: 

/* nRow 

/ * nCol 

/* nWidth 

/* nHeight 

/* nAttr 

/* i 

/* Return Value: None • 

/mitimtmmmmmmmmmitmmmmtmmmttmmmm/ 



- Row of the screen rectangle 

- Column of the screen rectangle 

- Width in columns of the screen rectangle 

- Number of rows in the screen rectangle 

- Color attribute used to clear screen rectangle 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



void pascal ScrClearRect ( nRow , nCol, nWidth, nHeight, nAttr) 
short , nRow, nCol, nWidth, nHeight, nAttr; 



auto 



union REGS 



nAttr 
r . h . ah 
r . h . al 
h . bh 
h . ch 
h . cl 
h.dh 
h.dl 



ScrCvtAttr ( nAttr ) ; 

(BYTE) BIOS_VID_SCROLL_UP; 



0; 

(BYTE) nAttr; 

( BYTE ) ( nRow - 

(BYTE) ( nCol - 
(BYTE) (nRow + 
(BYTE) ( nCol + 
int86 (BIOS_VID_INT, &r, 
return ; 



i) ; 
i) ; 

nHeight - 2) 
nWidth - 2 ) ; 
&r ) ; 



O 

ERIC 



mmmmmmmmmmmtmmmmmmmmmmmmmm/ 
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It SerSaveSect - This routine will save a screen rectangle in a caller */ 

supplied buffer area. nRow , nCol define the row and 
column of the upper left corner of the rectangle. 

Row of the screen rectangle 
Column of the screen rectangle 
nWidth - Width in columns of the screen rectangle 

- Number of rows in the screen rectangle 
Buffer used to store the saved screen rectangle 

e: None 

/mmmmmmmmmmmmmmmmmmmmmmmmm/ 



/* 




/* 




/* 


Farms : 


/* 


nRow 


/* 


nCol 


/* 


nWidth - 


/* 


nHeight 


/* 


pBuf 


/* 




/* 


Return Val 



void pascal ScrSaveRec t ( nRow 
short nRow, nCol , 

char *pBuf ; 

{ 

register unsigned 
register unsigned 
auto unsigned 

auto unsigned 

auto char far 



nCol , nWidth, nHeight, pBuf) 
nWidth, nHeight; 

uNumRows ; 
uColLen ; 
uScrOf s ; 

uBufSeg, uBufOfs; 

* f pBuf ; 




} 



uColLen = nWidth * 2; 

uScrOf s = ((nRow - 1) * SCR_BYTES_PER_ROW ) + (nCol - 1) * 2; 

fpBuf. = (char far * ) pBuf ; 
uBufSeg = FP_SEG( fpBuf ) ; 
uBufOfs = FP_OFF( fpBuf ) ; 

ScrEnableVideoCGA( CGA_DISABLE ) ; 

for (uNumRows = nHeight; uNumRows > 0; --uNumRows) 

{ 

movedata( uScrSeg , uScrOfs, uBufSeg, uBufOfs, uColLen); 
uScrOf s += SCR_BYTES_PER_ROW; 
uBufOfs += uColLen; 

} 

Sc r Enable Vi deoCGA ( CGA_ENABLE ) ; 
return ; 



/ttmtmmtmtmmtmmmmmmttmmmmttmmimmmt/ 



O 

ERIC 



/* 


ScrRes tore 


/* 




/* 




/* 




/% 




/* 




/* 




/* 


Parms : 


/* 


nRow - 


/* 


nCol 


/* 


nWidth - 


/* 


nHeight 


/* 


pBuf . 


/* 




/* 


Return Val 



a previously saved caller buffer. nRow and nCol 
define the upper left corner of the rectangle on 
the screen and are not required to be the same 
coordinates used in the save call. nWidth and 
nHeight should remain unchanged from the save call 
but are not required to do so. 



- Number of rows in the screen rectangle 



None 



'om 



of 11 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 
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/ 



mimmmmmmmm 



tmmmttmmttmmmttuutmmt 



tttti 



void pascal ScrRestoreRect ( nRow , nCol , nWidth, nHeight, 
short nRow, nCol , nWidth, nHeight: 



char *pBuf ; 

{ 

register unsigned 
register unsigned 
auto unsigned 

auto unsigned 

auto char far 



uNumRows ; 
uColLen ; 
uScrOf s ; 

uBuf Seg , uBufOfs; 
* f pBuf ; 



pBuf ) 




} 



uColLen = nWidth * 2; 

uScrOf s = ((nRow - 1) * SCR_BYTES_PER_ROW ) + (nCol - 1) * 2; 
fpBuf = (char far *) pBuf ; 
uBufSeg = FP_SEG( fpBuf ) ; 
uBufOfs. = FP_OFF( fpBuf ) ; 

ScrEnableVideoCGA( CGA_DISABLE ) ; 

for (uNumRows = nHeight; uNumRows > 0; —uNumRows) 

{ 

movedata ( uBufSeg , uBufOfs, uScrSeg , uScrOf s , uColLen); 
uScrOf s += SCR_BYTES_PER_ROW; 
uBufOfs += uColLen ; 

} 

ScrEnableVideoCGA ( CGA_ENABLE) ; 
return ; 



/nmmmmmmmmmmmmnmmmmitmmmmmmm/ 

/* ScrScrol IRectUp - Scrolls a screen rectangle up the requested nuaber */ 

/ * of lines . 

/* Parms : 

/ * nRow - Row of the screen rectangle 

/ * nCol - Column of the screen rectangle 

/* nWidth - Width in columns of the screen rectangle 

/* nHeight - Number of rows in the screen rectangle 

/ * nNoRows - Number of rows to scroll 

/* nAttr - Color attribute to fill blank line on bottom 

/* . 

/* Return Value: None 

/tiitititiiiiittititittitiiiiiiittttiitiiiiitttiimimimiinmmmttt/ 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



void pascal ScrScrollRectUp ( nRow , nCol, nWidth, nHeight, nNoRows, 
short nRow, nCol , nWidth, nHeight, nNoRows, nAttr’ 

{ 




auto 




union REGS r; 




nAttr 


“ 


ScrCvtAttr ( nAttr ) ; 




r . h . ah 


— 


BIOS_VID_SCROLL_UP ; 




r . h . al 


- 


(BYTE) nNoRows; 




r . h . bh 


— 


(BYTE) nAttr; 




r . h . ch 


- 


(BYTE) (nRow - 1 ) ; 




r . h . cl 




(BYTE) ( nCol - 1 ) ; 




r . h . dh 




(BYTE) (nRow + nHeight 


- 2) 


r . h . dl 




(BYTE) (nCol + nWidth - 


2) ; 



nAttr ) 



72 
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} 



int86(BIOS_VID_INT, &r, &r ) ; 
return ; 



/tnmmmmnnmmmmmmmttmtmmmmmmtmmtm/ 

/* ScrSerol lEectDown - Scrolls a screen rectangle up the requested nuaber */ 

/ * of lines . t / 

/* Parms : 

/* nRow - Row of the screen rectangle %/ 

/* nCol - Column of the screen rectangle */ 

/ * nWidth - Width in columns of the screen rectangle */ 

/* nHeight - Number of rows in the screen rectangle */ 

/* nNoRows - Number of rows to scroll */ 

/* nAttr - Color attribute to fill blank lines on top */ 

/* */ 
/* Return Value: None " */ 

/ititittttititittittitiiiimmimimtitmmmmmtmmmtmtm/ 



void pascal ScrScrollRectDown ( nRow , nCol , nWidth, nHeight, nNoRows, nAttr) 
short nRow, nCol , nWidth, nHeight, nNoRows, nAttr; 

{ 

auto union REGS r; 




} 



nAttr 
r . h . ah 
r . h . al 
r . h . bh 
r . h . ch 
r . h . cl 
r . h . dh 
r . h . dl 
int86 (BIOS 
return ; 



ScrCvtAttr ( nAttr ) ; 
BIOS_VID_SCROLL_DOWN ; 
(BYTE) nNoRows; 
nAttr ; 

( nRow 
( nCol 
( nRow 
( nCol 



( BYTE ) 

(BYTE) 

( BYTE ) 

(BYTE) 

( BYTE ) 

VID_INT, &r , 



i) ; 
i) ; 

nHeight 
nWidth 
&r ) ; 



2 ) ; 
2 ) ; 



/mmmmmmmmmmmmmmmmmmmmmmmmm/ 

/* ScrSetCursorPos - This routine will position the cursor to an absolute */ 

/* screen coordinate using the BIOS video services. 

/* Parms: 

/* nRow - Absolute screen row 

/* nCol - Absolute screen column 

/* 

/* Return Value None 

/mmmmmmmummmummmmmmmmmummmm/ 



void pascal 
short 



auto 



ScrSetCursorPos ( nRow , 
nRow, nCol ; 

union REGS r; 



nCol ) 




r . h . ah 
r . h . dh 
r . h . dl 
r . h . bh 



BIOS_VID_SET_CURSORPOS 
( BYTE ) ( nRow - 1 ) ; 

(BYTE) ( nCol - 1 ) ; 
(BYTE) nCurrActivePage 




*/ 

*/ 

*7 

*/ 

*/ 

*/ 
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} 



int86 ( BIOS_VID_INT , &r, &r); 
return ; 



/tttitttttttttttitmmmmtmtmttmtmmmtmmmutttmmtt/ 

/* ScrGetCursorPos - This routine will return the current absolute */ 

/* cursor position. */ 

/* Parms : * / 

/* nRow - Pointer to location to save current row */ 

/* ' nCol - Pointer to location to save current column */ 

/* */ 
/* Return Value: None %/ 

/mtmmmmmmtmmtmmmmmmmmtmtmmmmmt/ 



void pascal ScrGetCursorPos ( nRow , nCol) 
short *nRow, *nCol ; 

{ 

auto union REGS r; 

r.h.ah = BIOS_VID_GET_CURSORPOS ; 
r.h.bh = (BYTE) nCurrActivePage ; 
int86 (BIOS_VID_INT, &r , &r ) ; 

*nRow = r.h.dh + 1; 

*nCol = r . h . dl + 1 ; 




/tmtmmmtmmmmmmtmmtmmmtmmmimmimmt/ 

/ * ScrCusrosOn - Enables the screen cursor. %/ 

/* */ 
/* Parms: None %/ 

/* i t/ 

/* Return Value: None • * / 

mm mm 



void pascal ScrCursorOn ( ) 

{ 

auto union REGS r; 

r.h.ah = BIOS_VID_SET_CURSORTYPE ; 
r.x.cx = uCsrType; 
int86 (BIOS_VID_INT, &r , &r ) ; 
return ; 

} 



*/ 

*/ 

*/ 

*/ 

*/ 



/mmmummmmmtmmmmmummumtmmmtmmm/ 



/* 


ScrCusrosOf f 


- Disables 


/* 

/* 


Parms : 


None 


/* 

/* 


Return Value : 


None 



luttnttuuututtuuuuttuutuuttttutnutntuutttuutttuttttui 



O 

ERIC 
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void pascal 

{ 

auto 



ScrCursorOf f { ) 
union REGS 



r ; 



r.h.ah = BIOS_VID_SET _CURSORTYPE ; 
r.x.cx = OxOfOO; 
int86 { BIOS_VID_INT , &r, &r ) ; 
return ; 

} 



/mmmmmmmtmmmmmmmmmmmmmmmmmu/ 

/* ScrTextOut - This function uses the BIOS write character and attribute </ 



/* 




service routine to display a string within a 


/* 




passed nCount is used to limit a string from 


/* 




a window boundry. 


/* 


Parms : 


— 


/* 


pStr 


- Pointer to the string to be displayed 


/* 


nAttr 


- Color attribute used to display string 


/* 


nCount 


- Maximum number of characters to dispalay 


/* 




/* 


Return Value: None 



/mtmmmmmmmssmmtmmmmmtmmmmmmmtm/ 



window. The 
overflowing 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

%/ 

*/ 



void pascal ScrTextOut ( pStr , 
register char *pStr; 

short nAttr, 

{ 



nAttr, nCount ) 
nCount ; 



auto short nRow, nCol ; 

auto union REGS r, rl; 



ScrGetCursorPos ( &nRow , &nCol); 
nAttr = ScrCvtAttr ( nAttr ) ; 
r.h.ah = BIOS_VID_WRITE_CHATTR ; 
r.h.bh = (BYTE) nCurrActi vePage ; 
r.h.bl = (BYTE) nAttr; 
r.x.cx = 1 ; 

while (*pStr && nCount-- > 0) 

{ 

ScrSetCursorPos ( nRow , nCol++); 

r . h . al = *pStr++ ; 

int86 (BIOS_VID_INT, &r, &rl); 

} 

return ; 



/liiititttiiiiiiimmimmmimimimmimmiitiimimitmim/ 




/* ScrDrawRect 
/* 

/* 

/* 

/* Parras: 

/ * nRow - 

/ * nCol 

/* nWidth - 



- This routine is used to draw borders around a screen 
window. The passed parameters define the rectangle 
being used by the window as well as the border color 
and type . 

Top row of screen border 
Left column of screen border 
Column width of the window 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 
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/* nlleight - Number of rows in the window 

/* nColor - Color attribute for the window border 

/* nType - Type of border to be displayed 

/* 

/* Return Value: None 

/mtmmmsmmmtmmmmmmmmmmmmmmmmm/ 



void pascal ScrDrawRect ( nRow , nCol, 
short nRow, nCol , nWidth, 

{ 

register short i; 

auto union REGS r, rl; 

static BOXTYPE BoxTypes (] = 

{ 



{ 


32 , 


32 , 


32 , 


32 , 


32, 


32 


{ 


213 , 


212 , 


184, 


190, 


179, 


179 


{ 


214, 


211 , 


183 , 


189, 


186, 


186 


{ 


201 , 


200 , 


187, 


188, 


186, 


186 


{ 


218 , 


192 , 


191 , 


217 , 


179, 


179 


{ 


219 , 


219 , 


219, 


219 , 


219, 


219 



} ; 



nWidth, nlleight, nColor, nType) 
nHeight , nColor, nType; 



32 , 


32 


} , 


/* 


N0_B0X 






205 , 


205 


} , 


/* 


DBL_LINE 


TOP 


BOTTOM 


196 , 


196 


} , 


/* 


dbl_line 


SIDES 


205 , 


205 


} , 


/* 


DDL LINE 


ALL 


SIDES 


196 , 


196 


} , 


/* 


SNGL LINE ALL 


SIDES 


223 , 


220 


} 


/* 


GRAPHIC 


BOX 


*/ 



*/ 

*/ 

*/ 

*/ 

*/ 



if (nType < 0 jj nType >= MAXDIM ( BoxTypes ) ) 
return ; 

if (nWidth < 2 i j nHeight < 2) 
return ; 

nColor = ScrCvtAttr ( nColor ) ; 

/* Draw upper left corner */ 

ScrSetCursorPos ( nRow , nCol); 

r.h.ah = (BYTE) BIOS_VID_WRITE_CHATTR ; 

r . h . al = (BYTE) BoxTypes [nType J. cUpperLef t ; 

r.h.bh = (BYTE) nCurrActivePage ; 

r.h.bl = (BYTE) nColor; , 

r . x . cx = 1 ; • 

int86 (BIOS_VID_INT, &r, &rl); 



/* Draw upper right corner */ 

ScrSetCursorPos ( nRow, nCol + nWidth - 1); 
r.h.al = (BYTE) BoxTypes [ nType ]. cUpperRight ; 
int86 ( BIOS_VID_INT , &r , &.rl); 



/* Draw lower left corner */ 

ScrSetCursorPos ( nRow + nHeight - 1, nCol); 
r.h.al = (BYTE) BoxTypes T nType ]. cLowerLeft ; 
int86 (BIOS_VID_INT, &r, &rl); 



/* Draw lower right corner */ 

ScrSetCursorPos ( nRow + nHeight - 1, nCol + nWidth - 1); 
r.h.al = (BYTE) BoxTypes [ nType ]. cLowerRight ; 
int86(BIOS_VID_INT, &r, &rl); 

if (nHeight > 2) 

{ 

/* Draw left side line */ 

r.h.al = (BYTE) BoxTypes [nType ]. cLeft ; 
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for 

{ 



} 



(i = 1; i <= nHeight - 

ScrSetCursorPos ( nRow + 
int86(BIOS_VID_INT, &r 



2; ++i ) 

i , nCol ) ; 
&r 1 ) ; 



/* Draw right side line */ 

r.h.al = (BYTE) BoxTypes [ nType ] . cRight ; 

for (i = 1; i <= nHeight - 2; ++i) 

{ 

ScrSetCursorPos ( nRow + i, nCol + nWidth - 1); 
int86 (BIOS_VID_INT, &r , &rl); 

1 

} 



if (nWidth > 2) 

{ 

/* Draw top line */ 

ScrSetCursorPos ( nRow , nCol + 1); 
r.h.al = (BYTE) BoxTypes [ nType ]. cTop ; 
r . x . cx = nWidth - 2; 
int86(BIOS_VID_INT, &r , &rl); 




} 



/* Draw bottom line */ 
ScrSetCursorPos ( nRow + nHeight - 
r.h.al = BoxTypes [ nType 1 . cBottom ; 
int86(BIOS_VID_INT, &r , &rl); 



1 , 



nCol + 1 ) ; 



return ; 

} 



/ttmttttttmttitttimimttttmitmmtmmtmmtmitiimtttmi/ 

/* Scrlnitialize - Determine type of video card and init global data. 
/* 

/* Parms : None 

/* 

/* Return Value: None 

/mmmmmmmmmmmmmmmmmmmmmmmmm/ 



*/ 

*/ 

*/ 

*/ 

*/ 



void pascal Scrlnitialize ( ) 

{ 

auto union REGS r; 




r.h.ah = BIOS_VID_GET_CURSORPOS ; 
r.h.bh = (BYTE) nCurrAct i vePage ; 
int86(BIOS_VID_INT, &r , &r ) ; 
uCsrType = r.x.cx; 

r.h.ah = BIOS_VID_GET_CRTMODE ; 
int86(BIOS_VID_INT, &r , &r); 
nScrCols = r.h.ah; 

nCurrActi vePage = r.h.bh; 
if (7 == r.h.al) 

{ 
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uScrSeg = OxbOOO ; 
nVideoCard = VIDEO_MONO; 
return ; 

} 

r.h.ah = BIOS_VID_SET_CRTMODE ; 
r . h . al = 3 ; 

int86(BIOS_VID_INT, &r , &r); 
uScrSeg = 0xb800; 
nVideoCard = VIDEO_CGA; 
return ; 






ERIC 
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#include<conio . h> 

# include < string . h> 

# include < win . h> 

# include < keyboard . h> 



#def ine MAX_STR LEN 80 



/ ^ ^ ^ ^ ^ ^ ^ ^ 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 ^ 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 / 
/* FUNCTION get_key 

waits for a keystroke then gets a keystroke scan code 
from the keyboard buffer 



/* REMARKS 
/ * 

/* INPUT 
/* OUTPUT 
/* 

/* 

/* CALLED BY 
/* CALLS 



*/ 



the value of the key hit, if the key was a function * / 
key of any kind, then the negative of the scan code is*/ 
returned 
get_str 

msg_of_day, update_clock */ 



*/ 

*/ 

*/ 



*/ 

*/ 



/***********t***************tttt***t *********** *****t***t*** *********/ 



int get_key ( void ) 

{ 

int ch; e 



ch = getch ( ) ; 
if (ch == 0) 

ch = -getch ( ) ; 

return ch; 



} /*get_key*/ 



/********************* *************t*t********************tt*********/ 
/* FUNCTION get_str 

Gets a string from the keyboard. To exit either press*/ 
RETURN, or ESC. This function supports the backspace.*/ 



/* 

/* 


REMARKS 


/* 

/* 

/* 


INPUT 


/* 




/* 




/* 


OUTPUT 


/* 


CALLED : 


/* 


CALLS 



int lenstr : - length of the string to input 

- if lenstr > 80 then lenstr = 80 
int x & y : x and y screen locations to print the 

string. as it is being typed in 
int *last_key: the last keystroke 

the string typed in 



*/ 

*/ 



*/ 

*/ 



*/ 



*/ 

*/ 



ge t_key 

/tttttttt *********** **tt*tt*****t ******* t*****tt ********** tttt*****t*/' 

char *get_str (HWND wh , int lenstr, int x, int y, int *last key) 

{ ~ 

char s t r [ MAX_STR_LEN ] = " " ; 

char ch=0; 

short tx=0, ty=0; 

int num_letters=0 ; 



*/ 



*/ 



if (lenstr > MAX_STR_LEN) 
lenstr = MAX_STR_LEN; 



O 

RJC 



for (tx=x;tx<=(x+lenstr-l) ;tx++) /* clear out the space about to be typed in*/ 
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WinSet Cursor Pos(wh, 
cprintf ( " " ) ; 

} / *for* / 



y» tx); 



WinSetCursorPos ( wh , y, x); 

while (ch != UP && ch != DOWN && ch != ESC &.& ch != RETURN) 

{ 

ch = get_key( ) ; 

*last_key = ch ; 

if (ch == BACKSPACE \ \ ch >= SPACEBAR && ch <= TILDA) 

{ 

if (ch == BACKSPACE) 

{ 

if ( num_letters-- < 1) 

num_letters =0; /* backup internal cursor 1 element*/ 

str [ num_letters ] = 0; /* assign end of string */ 



ScrGetCursorPos ( &ty , &tx); 

if ( tx == 1) /* Handles backing up past the */ 

{ /* beginning of a line.*/ 

tx = 81; 
ty — ; 

} / * i f * / 



ScrSetCursorPos ( ty , 
if ( strlen ( str ) ) 
cprintf ( " "); 

} /*if */ 



tx- 1 ) 



else 

if (num_letters < lenstr && ch>= 

{ 



str [ num_letters++ ] = ch; 
str [ num_letters ] = 0; 

> / * i f * / 

> / * i f * / 

WinSetCursorPos ( wh , y, x); 
cprintf ( "%s" , str); 

} / *whi le * / 



SPACEBAR &&. ch < =TI LDA ) 



if (ch != ESC && ch != DOWN && ch != UP) 
return str; 
else 
return 

} /*get_str*/ 




88 



define DEBUG 1 



‘ include < coni o . h> 
r include < string . h> 
i include < alloc . h> 
finclude< process . h> 
r include < dir . h> 
Finclude<stdio . h> 
r include' 'win . h" 
finclude"pointsht . h” 

7 include "keyboard . h" 
f include "student . h" 
idefine BLANK STR LEN 



8 



char * program _root _di r ; /* sts package root directory */ 

void clear_student_space ( void ) ; 

int put_message ( char *); 

void main_menu ( void ) ; 

int get_f ilenames ( char *, char *); 

void title_screen ( void ) ; 



^define ADD_STUDENT 2 
#def ine SELECT_STUDENT 3 
#def ine DELETE_STUDENT 0 
#def ine EDIT_STUDENT 1 
#def ine MAX_STUDENTS 20 
ftdef inamN£ME_LEN 21 



/* max number of students*/ 



char 


*age ; 


char 


*diagnos i s ; 


char 


*person_l ; 


char 


*person_2 ; 


char 
} ; 


* location ; 


truct 


student_type student ; 



/*******************************************************************/ 



/* FUNCTION f ree_all %/ 
/* REMARKS frees all allocated memory space allocated by malloc */ 
/* INPUT int num_ts - number of main menu items */ 
/* OUTPUT */ 
/* CALLED BY main */ 
/* CALLS */ 



/%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% / 

void free_all(int num_tf 
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def ine 



DEBUG 



1 



"include < coni o . h> 

"include <string . h> 

# include < alloc . h> 

# include < process . h> 
#include<dir . h> 

# include <stdio . h> 

# include "win . h " 

# include" point sht . h" 

# include "keyboard . h" 
#include"student . h" 
"define BLANK STR LEN 8 ” 



char *program_root_dir ; /* sts package root directory */ 

i 

void clear_s tudent_space ( void ) ; 

int put_message ( char *); 

void main_menu ( void ) ; 

int get_f i lenames ( char *, char *); 

void t i tle_screen ( void ) ; 



#def ine 
"def ine 
#def ine 
"def ine 
#def ine 
#def ine 
#def ine 
#def ine 
#def ine 
#def ine 
#def ine 
#def ine 
#def ine 



ADD_STUDENT 2 
SELECT_STUDENT 3 
DELETE_STUDENT 0 
EDIT_STUDENT 1 

MAX_STUDENTS 20 /* max number 

NAME_LEN 21 

AGE_LEN 3 

DIAG_LEN 24 

ADD_ST "Add Student" 

SELECT_ST "Select Student" 

DEL_ST "Delete Student" 

EDIT_ST "Edit Student Description" 
EXIT_ST "Exit" 



of 



students * / 



short message_color = WHITE ! REV_BLUE ! HI_INTENSITY ; 
short warn ing_co lor = WHITE ! REV_RED ! HI_INTENSITY ; 
short win_color = WHITE ! REV_GREEN ! HI_INTENSITY ; 
short bdr_color = WHITE ! REV_GREEN ! HI_INTENSITY ; 
short reverse_color = WHITE ! HI_INTENSITY ; 
short EXPLODE = TRUE; 

FILE * f open ( ) ; 



struct 


text_type *textl; 


struct 


text_type *name_list ; 


struct 


student_type { 


char 


♦name ; 


char 


♦ age ; 


char 


♦^diagnosis ; 


char 


♦ person_l ; 


char 


*person_2; 


char 


*locat ion ; 
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struct student_type student; 



/* FUNCTION free_all %/ 
/* REMARKS frees all allocated memory space allocated by malloc */ 
/* INPUT int num_ts - number of main menu items */ 
/* OUTPUT 



main 



*/ 



/* CALLED BY 
/* CALLS 



void free_all(int num_ts ) 

{ 

int c ; 



*/ 

*/ 



f lushall ( ) ; 
fcloseall ( ) ; 
for ( c=0 ; c <num_ts ; C++ ) 
free(textl[c] .str) ; 




} 



for ( c = 0 ; c <MAX_STUDENTS ; C + + ) 
f ree ( name_list [c ] .str) ; 
free ( student . name ) ; 
free ( student . age ) ; 
free ( student . diagnosis ) ; 
f ree ( student . person_l ) ; 
f ree ( student . person_2 ) ; 
free ( student . location ) ; 
free ( text 1 ) ; 
f ree ( name_list ) ; 



/* 


FUNCTION 


/* 


REMARKS 


/* 




/* 


INPUT 


/* 




/* 




/* 




/% 




/* 


OUTPUT 


/* 


CALLED BY 


/* 




/* 




/* 


CALLS 



wtext_out 

Prints text to a window at a specified row, col and 
color 

HWND wh - window handle to print to 

int col - column of window to print at 

int row - row of window to print at 

char *str - string to print 



open_andjprint_student_window , display_st_inf o str, 
create_add_student_window , add_or_edi t_student7 
delete_or_edi t_student 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



int color - color attributes for the string to print */ 



*/ 

*/ 

*/ 

*/ 



*/ 



void wtext_out ( HWND wh , int col, 
{ 



} 



WinSetCursorPos ( wh , 
WinTextOut ( wh , str, 



row, col ) ; 
color ) ; 



int row, char *str, 



int color) 




.83 



11/07/1991 



STUDENT. C 



Page 3 of 20 



/* FUNCTION 
/* REMARKS 
/ * 

/* INPUT 
/* OUTPUT 
/* CALLED BY 
/* CALLS 



put_warning_msg 






*/ 




Opens a red and white 


window at 


the center of the 


*/ 




screen, and prints a 


message 




*/ 




char *str - string or 


message to 


print 


*/ 


*/ 


save_student_inf o 






*/ 


get_key 






*/ 



void put_warning_msg ( char *str) 

{ 

HWND wh ; 

wh = Wir.CreateWindow( 11 , 10, 60, 3, warning_color , DBL_LINE_SIDES , warningjjolor, EXPLODE); 
WinCenterText ( wh , 1, str, warning_color ) ; 
get_key( ) ; 

WinDestroyWindow ( wh ) ; 

} 



/* 


FUNCTION 


ass ign_menu_at tr ibs 


% / 


/* 


REMARKS 


assigns the menu attributes for the main menu to 


*/ 


/* 




allow for point and shoot routine 


*/ 


/* 


INPUT 






/* 


OUTPUT 






/* 


CALLED BY 


main 


*/ 


/* 


CALLS 


al locat e_text_ob j ect 


*/ 



*/ 

*/ 



void assign_menu_at.tr ibs ( void ) 

{ 

int first_row = 2; 
int lcol = 2; 



allocate_text_ob ject ( text 1 , 
allocate_text_object ( text 1 , 
allocate_text_ob.ject ( textl , 
allocate_text_ob ject ( textl , 
allocate_text_object (textl , 



0, ADD_ST , lcol, first_row); 

1, DEL_ST , lcol, f irst_row+2 ) ; 

2, EDIT_ST, lcol, f irst_row+4 ) ; 

3, SELECT_ST , lcol, first_row+6) 

4, EXIT_ST , lcol, f irs t_row+ 10 ) ; 



} 






/* FUNCTION 
/* REMARKS 
/* 

/* INPUT 
/* 

/* OUTPUT 
/* CALLED BY 
/* CALLS 



open_and_pr int_student_window 
opens and prints the main window of the 
program 

HWND *wh - pointer to the window handle 
int num_ts - number of menu items 

main 

wtext out 



student 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



*/ 



ERIC 



/*tttttttttttttttttttttttt*t********t*t*t*****t*tt*t**t****tt*t***tt/ 
void open_and_pr int_student_window ( HWND *wh, int num_ts ) 



*/ 
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{ 

int c ; 

Jvh = WinCreatetf indow( 7 , 15, 50, 14, win_eolor, DBLLIHBJLL JIBES, bdr_color, EXPLODE); 

WinCenterText ( *wh , 0, "Student", win_color); 
for ( c = 0 ; c < num t s ; C + + ) 

wtext_out ( *wh , text 1 [c ] • col , textlfc] .row, textl[c]*str, win_color ) ; 

} 



/ttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttt/ 



/* FUNCTION create_add_student_window */ 

/* REMARKS creates and prints the textl for the attribute input */ 

/* INPUT HWND *st__wh - pointer to the student window */ 

/* int lcol - left column the attribute names will be */ 

/* printed at */ 

/* OUTPUT */ 

/* CALLED BY add_or_edi t_s tudent */ 

/* CALLS wtext out */ 



/ttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttt / 
void create_add_student__window( int OPERATION, HWND *st_wh, int lcol) 



*st_wh = WinCreateWindowt 7 , 15, 50, 18, win color, DBL_LINB SIDES, bdr color, EXPLODE); 
switch (OPERATION) 

{ 



case ADD_STUDENT : WinCenterText ( *st__wh , 0, 

break ; 

case EDIT_STUDENT : WinCenterText ( *st_wh , 0, 

break ; 



} /*switch*/ 



’’Add Student”, 
’’Edit Student” 



win_color ) ; 
win_color ) ; 



wtext_out ( *st_wh , 
wtext_out ( *st_wh , 
wtext__out ( *st_wh , 
wtext_out ( * st_wh , 
wtext_out ( * s t_wh , 
wtext_out ( *st_wh , 
wtext_out ( *st_wh , 
wtext_out ( *st_wh , 
wtext_out ( *st_wh , 
wtext_out ( * s t_wh , 
wtext _out( * st_wh , 
wtext_out ( *st_wh , 
wtext_out ( *st_wh , 
wtext_out ( *st_wh , 



lcol, 2, "Name: [" , win_color); 
lcol+5+NAME_LEN, 2, M ] ” , win_color); 
lcol, 4, "Age: [" , win_color ) ; 
lcol+4+AGE_LEN , 4, ”]”, win_color); 
lcol, 6, "Diagnosis: [" , win_color); 
lcol+10+DIAG_LEN, 6, ”]”, win_color); 
lcol, 8, "Person 1:[”, win_color); 
lcol+9+NAME_LEN, 8, " ] ” , win_color); 
lcol, 10, "Person 2:[", win_color); 
lcol+9+NAME_LEN , 10, win_color); 

lcol, 12, "Location win_color); 

lcol+DIAG_LEN+l 0 , 12, "]", win_color); 

lcol, 14, "Exit and Save", win__color); 
lcol, 16, "Select Student", win_color); 



/* clear out new space */ 

clear_student_space ( ) ; 

} 

/ttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttt/ 
/* FUNCTION display_st_info_str */ 



O 
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/* 


REMARKS 


displays the individual attributes of each student 


*/ 




/* 


INPUT 


HWND wh - window handle to print to 


* / 




/* 




char *str - character string to print 


*/ 




/* 




char *blank - blank string 


*/ 




/* 




int col - column to print at 


* / 




/* 




int row - row to print at 


*/ 




/* 


OUTPUT 






*/ 


/* 


CALLED BY 


display_student_inf o 


*/ 


/* 


CALLS 


wtext_out 


*/ 



/ *******************************************t**************t ******** / 



void display_st_info_str ( HWND wh , char *str, char *blank, int col, int row) 

{ 

if (strcmpfstr, NULL) && strcmpl str , 

wtext_out ( wh , col, row, str, win_color); 
else 

wtext_out ( wh , col, row, blank, win_color); 

) 



/*******************************************************************/ 



/* 


FUNCTION 


display_s tudent__inf o 


* / 


/* 


REMARKS 


center for printing student attributes 


*/ 


/* 


INPUT 


HWND wh - window to print to 


*/ 


/* 

/* 


OUTPUT 


int lcol - column to print at 


* / 


/* 


CALLED BY 


add_or_edi t_student 


*/ 


/* 


CALLS 


di splay_s t__inf o_str 


*/ 



/***********************************t******************t**********tt/ 



*/ 



void display_student_info ( HWND wh , int lcol) 
{ 



display_st_inf o_str ( wh , 
display_st_inf o_str ( wh , 
display_st_inf o_str ( wh , 
display_st_inf o_str ( wh , 
display_st_info_str ( wh , 
display_st_inf o_str ( wh , 



student . name , BLANK_STR _LEN_8 , lcol+6, 2); 
student. age, ” ", lcol+5, 4); 

student. diagnosis , BLANK_STR_LEN_8 , lcol+11, 6); 
student . person_l , BLANK_STR_LEN_8 , lcol+10, 8); 
student, personj, BLANK_STR_LEN_8 , lcol + 10, 10); 

student . location , " ", lcol+11, 12); 



wtext_out ( wh , lcol, 14, "Exit and Save", win_color); 
wtext_out ( wh , lcol, 16, "Select Student", win_color); 



} 

/************************************************************ ittttttt y 



O 

R 



/* FUNCTION 
/* REMARKS 
/* 

/* INPUT 
/* 

/* OUTPUT 
/* CALLED BY 
/* CALLS 



write_str2f ile 
If the string exists then the string is written to 
a file, else a is written. 

FILE *fp - file to write to 
char *str - string to write 

update_student_list_f ile , save_student info 



**/ 




*/ 




*/ 




*/ 




*/ 




*/ 


*/ 


*/ 


*/ 
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/ttttttttt*tt***t*t********************ttttt*tt*t*t****tti.tt****t*tt/ 

void write_str2f ile ( FILE *fp, char *str) 

{ 

if ( str && strlen ( str ) > 0 ) 

{ 

fprintf(fp, "%s\n", str); 

}/*if */ 
else 
{ 

fprintf(fp, "-\n"); 

}/*else*/ 

} 

/*********************** ********************************************/ 



/* FUNCTION update_student_l ist_f i le */ 

/* REMARKS updates the student list - by adding the new name * / 

/* INPUT FILE *fp - file handle for the student list file * / 

/* OUTPUT */ 

/* CALLED BY save_student_inf o */ 

/* CALLS write_str2f ile , */ 






void update_student_list_f ile ( void ) 

{ 

FILE *fpl ; 

char str [ 10] =" " ; 

int FOUND = FALSE; 

/* check to ensure that the name does not exist in the file */ 
fpl = f open ( " studlist . dat " , "rt"); 

if (fpl) 

{ 

while (! f eof ( fpl ) && strcmp(str, "")) 

{ 

fgets(str, 10, fpl); 

str [ strlen( str ) -1 J = 0 ; 

i f ( ! strcmp ( str , student . name ) ) 

FOUND = TRUE; 

} /*while*/ 

} /*if */ 
fflush( fpl ) ; 
fclose ( fpl ) ; 



if ( ! FOUND) 

{ 

fpl = f open (" studli st . dat " , "at"); 

if (fpl) 

{ 

wri te_str2f i le ( f pi , student . name ) ; 

} /*if */ 
f f lush( fpl ) ; 
fclose ( fpl ) ; 
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} 



}/*if*/ 



/* FUNCTION 
/* REMARKS 
/* INPUT 
/* OUTPUT 
/* CALLED BY 
/* CALLS 



save_student_inf o 
saves the student attributes to a file 



add_or_edi t_student 
put_warinng_msg , write_str2f ile 



■■*****/ 
*/ 
*/ 



*/ 

*/ 



*/ 

*/ 



void save_student_info ( int OPERATION) 

{ 

FILE *fp ; 
char buf [ 100 ] ; 



strcpy(buf, program_root_dir ) ; 



if (mkdir( student. name) && OPERATION == ADD_STUDENT ) 

P u f warning msg ( "You need a unique name ! " ) j 

else 



{ 



strcat { buf , 
strcat ( buf , 
chdir ( buf ) ; 
fp = fopen( 

if (fp) 

{ 



"W") ; 

student 



name ) 



students.dat", "wt") 



write_str2f ile ( fp , 
wri te_str2f ile ( f p , 
wri te_str2f ile ( f p , 
write_str2f ile ( fp , 
write_str2f ile ( fp , 
write_str2f ile { fp, 
) /*if */ 



student . name ) ; 
student . age ) ; 
student . diagnosis ) ; 
student . person_l ) ; 
student . person_2 ) ; 
student . location ) ; 



f flush ( f p ) ; 
f close ( f p ) ; 

chdir ( program_root_dir ) ; 
if (OPERATION == ADD_STUDENT ) 
update_student_list_f i le ( ) ; 
) / * else * / 



) 



/*t*********************************t ********* t************t ******* 
/* FUNCTION clear_student_space 

/* REMARKS sets all student attributes to null 

/* INPUT 
/* OUTPUT 

/* CALLED BY create_add_student_window , add or edit student 
/* CALLS 



*/ 

*/ 

*/ 



*/ 




*/ 

*/ 

*/ 



O 

ERIC 



void 

{ 



clear_s tudent_space ( void ) 
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strcpy( student . name , 

strcpy ( student . age , 

strcpy ( student . diagnosis , ) ; 

strcpy ( student . person_l , 

strcpy ( student . person_2 , 

strcpy ( student . location , 

} 



/* FUNCTION 
/* REMARKS 
/* INPUT 
/* 

/* OUTPUT 
/* CALLED BY 
/* CALLS 



read_str_f rom_f ile 

reads a string from a file 

FILE *fp - file to read from 

char *str - pointer to string to return 

read_student data 



*/ 




*/ 




*/ 




*/ 


*/ 


*/ 


*/ 



void read_str_f rom_f i le ( FILE *fp, char *str) 

f gets ( str , NAME_LEN+1, fp); 
str [ strlen ( str ) -1 ] = 0; 

} 

/* FUNCTION check4null 

checks to see if the string is blank 
char *str - string to check 



*/ 

*/ 

*/ 



/* REMARKS 
/* INPUT 
/* OUTPUT 
/* CALLED BY 
/* CALLS 



read student data 



*/ 



void check4null ( char *str) 

{ 

if ( ! strcmp ( str , 
strcpy(str, NULL); 

} 




/* FUNCTION 
/* REMARKS 
/* INPUT 
/* OUTPUT 
/* CALLED BY 
/* CALLS 



read_student_data 
reads in the student data from the file */ 

char *name - the students name/directory to read from* / 



add_or_edi t_student 
read_str_f rom_f ile , read_student_data 



*/ 

*/ 



*/ 

%/ 



*/ 



void read_student_data ( char *name) 

{ 

FILE *fp; 

char str [NAME_LEN] =" " , buf[100]; 

strcpy (buf, program_root_dir ) ; 
strcat (buf , "\\" ) ; 
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strcat ( buf , 



name ) ; 



chdir ( buf ) ; 

fp = fopenj "students .dat" , "rt" ); 



read_str_f rom_f ile ( fp , str ) ; 
strcpy ( student . name , str); 
check4null (student . name ) ; 



read_str_from_f ile ( fp , str); 
strcpy ( student . age , str); 
check4null ( student . age ) ; 



read_str_f rom_f ile ( fp , str); 
strcpy ( student .diagnosis , str); 
check4null ( student . diagnosis ) ; 



read_str_f rom_f ile ( fp , str); 
strcpy( student .person_l , str); 
check4null ( student . person_l ) ; 

read_str_f rom_f ile ( fp , str); 
strcpy ( student . person_2 , str); 
check4nul 1 ( student . per son_2 ) ; 



read_str_f rom_f ile ( fp , str); 
strcpy ( student . location , str); 
check4null ( student . location ) ; 



f flush ( fp ) ; 
fclose ( fp ) ; 

chdir ( program_root_dir ) ; 

} 



/************************************* ******************************/ 
/* FUNCTION add_or_edi t_s tuden t % j 

/* REMARKS control^center for^adding a student to the STS system*/ 

*/ 



int OPERATION - ADD or EDIT 



char *name - name of student to edit 



*/ 



/* INPUT 
/* 

/* OUTPUT 

/* CALLED BY main %/ 

/* CALLS create add student_window , clear student space, %/ 

/* display_student_inf o , wtext_out, read_student data %/ 

/tttttttttttttttttttttxttttttttttttttttttttttttttttttttttttttttttttt/ ' 

char* add_or_edit_student ( int OPERATION, char *name) 

{ 

HWND st_wh; 

int lcol = 2, num_items = 7; 

char *str; 

int item = 0 ; 

int FINISHED = FALSE; 

int last_key = 0; 

int rc = 0 ; 



*/ 



O 

ERIC 



str = (char * ) malloc ( s izeof ( char ) * NAME_LEN ) ; 
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create_add_student_window( OPERATION , &st_wh, lcol); 
switch (OPERATION) 

{ 

case ADD_STUDENT : clear_student_space( ) ; 

break ; 

case EDIT_STUDENT : read_student_data ( name ) ; 

break ; 



} /*switch*/ 



f 




textattr ( win_color ) ; 
while ( ! FINISHED) 

{ 

display_student_info( st_wh , lcol ) ; 
switch ( item) 

{ 

case 0 : if (OPERATION != EDIT_STUDENT ) 

{ 



&last_key ) ) ; 



strcpy(str, get_str ( st_.wh , NAME_LEN-1, 

if (strcmplstr , NULL)) 

strcpy ( student . name , str) ; 

break ; 

} /*else*/ 
else 



item= 1 ; 



lcol + 6 , 



case 



case 




case 1 : strcpv(str, get_str ( st_wh , AGE_LEN- 1 , lcol+5, 4, &last_key)); 

if (strcmp(str, NULL)) 

strcpy ( student . age , str); 
break ; 

case 2 : strcpy(str, get_str ( st_wh ,, DIAG_LEN- 1 , lcol + 11, 6, &last_key)); 
if (strcmplstr, NULL)) 

strcpy ( student . diagnosis , str); 
break ; 

case 3 : strcpy(str, get_str ( st_wh , NAME_LEN- 1 , lcol+10, 8, &last_key ) ) ; 
if (strcmplstr, NULL)) 

strcpy ( student . person_l , str); 
break ; 

4 : strcpy ( str , get_str(st_wh, NAHB LEN- 1 , lcoltlO, 10, ilast key } } ; 

if (strcmpfstr , NULL)) 

strcpy ( student . person_2 , str); 
break ; 

5 : strcpy ( str, get_str(st_wh, DIAG_LEN-1, lcol + 11, 12, ilastkeyjj; 

if (strcmplstr, NULL)) 

strcpy ( student . location , str); 
break ; 

case 6 : wtext_out ( st_wh , lcol, 14, "Exit and Save", reverse color); 
last_key =0; 

while ( last bey !: DOWN 41 last_key !: RETURN 11 last ksy !: ESC 11 last_key !: UP) 
last_key = get_key( ) ; 



O 

ERLC 



if ( last_key = = ESC || last_key == RETURN) 
FINISHED = TRUE; 
break ; 
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case 



7 



wtext_out ( st_wh , lcol , 16, "Select Student", reverse color); 

last_key =0; ~ 

while ( last_key != RETURN && last_key != ESC && last_key != UP 
last_key = get_key( ) ; 



if ( last_key == ESC “ last_key == RETURN) 
FINISHED = TRUE; 
if (last_key == RETURN) 
rc = 1 ; 
break ; 



} / *switch*/ 



switch ( last_key ) 

{ 



case UP: 



case RETURN: 



case DOWN: 




} 



case ESC: 
/*switch*/ 



if (--item <0) 
item = 0 ; 
break ; 

if (++item > num_items) 
item = num_items; 
break ; 

if (++item > num_items ) 
item = num_items; 
break ; 

FINISHED = TRUE; 
break ; 



} /*while*/ 



WinDestroyWindow( st_wh ) ; 
if ( last_key == RETURN) 

save_student_info (OPERATION ) ; 

free ( str ) ; 
if ( rc ) 

return student . name ; 
else 

return " " ; 



} 



*/ 

*/ 



/* FUNCTION delete_f i le ' 

/* REMARKS if the file or wildcard exists, then the file is 
/* deleted without any screen output 

/ * INPUT char *file - file to delete ^ / 

ft OUTPUT the success of the deletion 

/* CALLED BY delete_data_f i les 

/* CALLS ' 



O 

ERiC 



int delete_f ile ( char *file) 

{ 



92 
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char del_cmd[] = 
char coramand[ 19] 
struct f fblk f ; 
int a=0: 



"del 



if ( f indf irst ( f ile , &f, a)==0) 

{ 



strcpy( command, del_cmd) ; 
strcat ( command , file); 
return system) command) ; 
}/*if*/ 



else 

return - 1 ; 

} 



/* FUNCTION delete_data_f i les */ 
/* REMARKS deletes a file from a path; handles wildcards t/ 
/% INPUT char *path - path to delete */ 
/* int color - color to display the message X / 
/% OUTPUT int - success or not %/ 
/* CALLED BY delete_student_named %/ 
/* CALLS 

/****xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx*xxxxxxxxxx*t/ 




int 

{ 



delete_data_f iles ( char *path, 



int color) 



int x=0, c; 

char str[ 100] ="" ; 

extern char directory [ 50 ][ 14 ] ; 



*/ 



strcpy ( str , program_root_dir ) ; 
strcat(str, "\\"); 
strcat(str, path); 



chdir ( str ) ; 

c = get_f ilenames ( "c : " , "*.dat"); 




c - = 2 ; 

textattr ( color ) ; 
while(x <= c) 

{ 

strcpy(str, directory[x] ) ; 
strcat ( str , ".*"); 

delete_f ile ( str ) ; 

x + + ; 

} /* while*/ 

delete_f ile ( " * . dab" ) I 
delete_f ile ( " # . vpc " ) ; 
delete_file( "null. * " ) ; 

for (x=0 ; x< =9 ; x++ ) 

{ 

sprintf ( str , "*.%ld*", x); 

delete_f ile ( str ) ; 



/* the address is 
when this functi 




blown 

on returns 



*/ 
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} 



} /*f or*/ 

delete_f ile ( "num . " ) ; 
chdir ( program_root_dir ) ; 

puts ( "Returning c"); 
return c ; 



/********xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxtxxxxxxxtxxxxxxxx/ 

/* FUNCTION delete_name_f rom_student_l ist %/ 

/ * REMARKS deletes a student name from the studlist.dat file */ 

/* INPUT char *name - name of student to delete */ 

/* OUTPUT %/ 

/* CALLED BY delete_student_named %/ 

/* CALLS ' 

/***XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/ 

void delete_name_from_student list(char *name) 

{ 

FILE *fpl, *fp2 ; 

char *sname ; /X student name read in from the file*/ 

fpl = fopen( "studlist.dat" , "rt" ) ; 
if (fpl) 

{ 

sname - (char * ) malloc (sizeof(char) * 9); 
strcpy ( sname , "qwerty\n" ) ; 
fp2 = f open (" temp . dat" , "wt") ; 
fgetsfsname, 9, fpl); 

while( ! feof ( fpl ) && strcmp ( sname , NULL)) 

{ 

sname [ strlen ( sname )- 1 ] = 0; 
if ( strcmp ( sname , name)) 

{ 

fputs(sname, fp2); 
fputc ( ’ \n ’ , fp2 ) ; 

} /*if */ 



*/ 



strcpy ( sname , NULL); 

. fgets ( sname , 9, fpl); 

}/* while*/ 
f flush ( f p2 ) ; 
f close ( f p2 ) ; 
remove ( "studlist.dat" ) ; 
rename ( "temp.dat" , "studlist.dat" ) ; 
} / * if */ 

f flush ( f pi ) ; 
fclose ( fpl ) ; 



O 

ERIC 



} 

/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/ 

/* FUNCTION delete_student_named ^ . 

/* REMARKS yes/no query to verify deletion of user %/ 

/* INPUT char *name - name of user to delete x/ 
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/* OUTPUT 

/* CALLED BY delete__or _edit_student 

/* CALLS delete_data_f iles , delete_name_f rom_student_l i s t 

/X****************x*************x********x**x********x********x***x* / 



X/ 

*/ 



void delete_student_named ( char *name) 

{ 

HWND w_wh; 
char *buf ; 

short DELETED = FALSE; 



%/ 



buf = (char *) malloc ( sizeof ( char ) * 50); 

'i/_wh : rfinCreateWindow( 1 1 , 10, 60, 5, warning_color, DBl_LINE_SIDES, warning color, EXPLODE); 

sprintf ( buf , "Deleting %s will’ erase all of % s’s files.", name, name); 
WinCenterText ( w_wh , 1, buf, warning_color ) ; 

WinSetCursorPos ( w_wh , 3, 19); 

if ( delete_data_f i les ( name , warning_color ) > -1) 

DELETED = TRUE; 
else 

DELETED = FALSE; 



printf ( "Returned from delete data files"); 
get_key ( ) ; 



Wi 



if 

{ 



nDestroyWindow ( w_wh 
(DELETED) 



rmdir ( name ) ; 



/* delete name from studlist.dat 
delete_name_f rom_student_list ( name ) ; 

} /*if */ 



file 



*/ 



f ree ( buf ) ; 

} 



/ XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX* / 



/% FUNCTION delete_or_edit_student x/ 
/X REMARKS command center for deleting or editing a student * / 
/* INPUT int OPERATION - DELETE or EDIT a student */ 
/X OUTPUT 

/X CALLED BY main %/ 
/X CALLS wtext_out , delete_student_named */ 



/xxxxxxxx****xxttxxxxxx*xxx*xxxxxxxxxxxtxxxxxxxxxxxxxxxxxxxxx%xxxx%x/ 

char* delete_or_edit_student ( int OPERATION) 

{ 

FILE *fp; 
char *str; 
int c , x ; 



HWND sl_wh; /^student list window handle*/ 

HWND w_wh ; /* warning window handle*/ 

char st_name [NAME_LEN+1 ] =" " ; 



str 



(char * )malloc( sizeof (char ) * NAME_LEN ) ; 



*/ 



fp = fopen( "studlist.dat' 
if (fp) 



’rt" ) 



ERiC 



11/07/1991 



STUDENT . C 



Pa.ee 15 of 2( 




while ( ! f eof ( f p ) && c < MAX_STUDENTS ) 

{ 

fgets ( name_list [ c ! .str, NAME_LEN+1, fp); 
if ( name_list f c ] . str ) 

{ 

name_list [c ] .str[strlen( name_list [c ] . str ) -1 1 =0 ; 

C + + ; 

} / * f or * / 

} / *while* / 
f flush ( fp ) ; 
fclose(fp); 



x = c - 1 ; 
c- = 2 ; 
if ( c>=0 ) 

{ 

sl_wh : WinCreateWindowf 1 , 1, 25, x+2, win_color, DBL_LINB_SIDES, bd; color, EXPLODE ) ; 

WinCenterText ( sl_wh , 0, "Student List", win_color); 
while(c >= 0) 

{ 



wtext__out ( sl_wh , 
name_list[c] .col 
name_list[c] .row 
c , 

} /*whi le* / 

} /*if */ 



c + 1 , 

2 ; 

c + 1 j 



name_list[cl .str, 



win_color ) ; 



if ( x > 0 ) 

{ 

w_wh : WinCreateWindowf 11, 10, 60, 3, nessage color, DBL LINE SIDES, aessaie color, EXPLODE ) ; 
switch (OPERATION) 

{ 



case DELETE_STUDENT : WinCenterText ( w_wh , 
"Point to the student to delete.", aessage_color) ; 
break ; 

case EDIT_STUDENT : WinCenterText ( w_wh , 
"Point to the student to edit.", aessage color); 
break; 



} /*switch*/ 



1 , 

1 , 





if (get_key( ) != ESC) 

{ 

WinDestroyWindow( w_wh ) ; 

c = point_and_shoot ( sl_wh , name_list, x); 

if (c>-l) 

switch ( OPERATION ) 

{ 



case DELETE_STUDENT : delete_student_named ( name_l ist [ c- 1 1 . str ) * 

break ; ’ 

case EDIT STUDENT : strcpyf st_naae , add or edi t_student ( ED IT_STUDEMT. name list(c-l) .str!) ; 

break ; 

} / * switch* / 

} /*if */ 

else s-sra 



11/07/1991 



STUDENT. C 



Page 16 of 2( 




Wi 



nDestroyWindow ( w_wh ) 



} /*if */ 
else 
{ 

put_message ( "No students found!"); 



} /*else*/ 



} /*if */ 
else 

put_message ( "No students found!"); 
f close ( f p ) ; 
f ree ( str ) ; 

WinDestroyWindow( sl_wh ) ; 
if ( strcmpf st_name , "")) 

return st_name; 
else 

return " " ; 

} 




/ 

/ 

/ 

/ 

/ 

/ 

/ 

/ 



***************************** 

* FUNCTION put_message 

* REMARKS puts a message 

* INPUT char *str - mes 

* OUTPUT 

* CALLED BY whether the user 

* CALLS get_key 

***************************** 



************************************ 

in a window 
sage 

wishes to continue or ESC 
************************************ 



* * / 
*/ 
*/ 
*/ 

*/ 

*/ 

**/ 



int put_message ( char *str) 

{ 

HWND wh ; 
int key; 



*/ 



wh : V inCreateWindowf 1 1 , 10, 60, 3, nessage_color, DBL_LIiJE_SIDES, aessage_color, EXPLODE) ; 
WinCenterText ( wh , 1, str, message_color ) ; " 

key = get_key( ) ; 

WinDestroyWindow ( wh ) ; 
if (key == ESC) 
return -1 ; 
else 

return 1 ; 






* FUNCTION display_student_list 

* REMARKS opens a window and displays a student list for 

* point and shoot 

* INPUT 

* OUTPUT 

* CALLED 

* CALLS 



*/ 

*/ 

*/ 



int - student id of the selected student * / 

BY selec t_s tudent * / 

point_and_shoot , put_message */ 

************************************************ ^tttt*******tt*t tt* / 
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int 

{ 



display_student_list ( void ) 



char *str; 
int c, x ; 
FILE *fp ; 
HWND sl_wh; 



str = (char * ) malloc ( si zeof ( char ) * 10); 

fp = fooen( "studlist .dat" , ”rt"); 
if (fp) 

{ 

c = 0 ; 

while (! feof ( fp) && c < MAX_STUDENTS ) 

{ 

fgets ( name_list [ c ] . str , NAME_LEN , f p ) ; 
if ( name_l ist [ c ] . str ) 

{ 

name_list [ c ] .str[strlen( name_list(c] . str ) -1 1 =0 ; 

C + + ; 

} / * f or* / 

} /*while*/ 



x = c - 1 ; 
c- = 2 ; 
if ( c> = 0 ) 




si 



{ 

wh : WinCreateWindowf 1 , 1, 25, 
WinCenterText(sl 
whilefc >= 0) 



x+2 , win co lor , 
wh , 0 , ’’ 



BBUIN8 .SIDES, bdr_color, EXPLODE); 
Student List", win_color); 



{ 



wtext_out ( sl_wh , 2, c+1, name_list[c] .str, win_color); 
name_l i st [ c ] . col = 2; 
name_list [ c ] . row = c+1; 
c — ; 

} / * whi le * / 

}/*if */ 



if ( x > 0 && put_message ( "Point to the desired student.") > -1) 

c = point_and_shoot ( sl_wh , name_list, x); 
else 

if (x <= 0) 

put_message ( "No students found!"); 

} /*if */ 
else 
{ 

put_message( "No students found!"); 
c = -1 ; 

} /*else*/ 



fclose ( fp) ; 
f ree ( str ) ; 

WinDestroyWindow ( sl_wh ) ; 
return c; 

} 
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/* FUNCTION 
/* REMARKS 
/* INPUT 
/* OUTPUT 
/* CALLED BY 
/* CALLS 



select_student 
prompts the user to select the student 
char *name - student’s name 



*/ 

*/ 

*/ 



ma in % / 

display_student_list */ 



void select_student ( char *name) 

{ 

int c ; 

char buf [100] ; 



*/ 



if ( ! strcmp ( name , "")) 

{ 



c = display_student_list ( ) ; 
if ( c> - 1 ) 

{ 



program_root_dir ) ; 
”\\" ) ; 

name_list[c-l ] . str) ; 



strcpy ( buf , 
strcat ( buf , 
strcat ( buf , 
chdir ( buf ) ; 
main_menu ( ) ; 

textattr ( reverse_color ) ; 
clrscr ( ) ; 

) /*if */ 

} / * i f * / 

else 



{ 

strcpy(buf, program_root_dir ) ; 
strcat ( buf , "\\”); 

strcat(buf, name_list [ c- 1 ] . str ) ; 
chdir ( buf ) ; 



main_menu ( ) ; 

textattr ( reverse_color ) ; 
clrscr ( ) ; 

}/*else*/ 



} 

void main(void) 

{ 

int item = 0; 
int c ; 

char st_name [NAME_LEN+ 1 ] = " " ; 

HWND wh ; 

int num_ts = 5; /% num items on main menu*/ 

#define buflen 80 
char buf [buflen]; 





#if ! DEBUG 

spawnlp ( P_WAIT , "bird", 0); 
#endi f 
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puts ( "Trying to initialize"); 
Winlnitialize ( ) ; 
puts( "Initialized" ) ; 



textattr ( reverse_color ) ; 
clrscr ( ) ; 
title_screen( ) ; 
puts( "Title Screen"); 

nasejist : (struct tezt typet) mallocf sizeof (struct text_type ) * MAXSTUDBUTS) ; 

text 1 = (struct text_type* ) malloc ( sizeof ( struct text_type) * num_ts ) ; 
student. name = (char *) malloc ( s i zeof ( char ) * NAME_L£N); 

student. age = (char *) malloc (sizeof (char) * AGE_LEN ) ; 
student . diagnosis = (char *) malloc ( s i zeof ( char ) * DIAG_LEN); 

student . person_l = (char *) malloc ( s i zeof ( char ) * NAME_LEN ) ; 
student .personj = (char *) malloc ( s i zeof ( char ) * NAME_LEN); 
student . location = (char *) malloc ( sizeof ( char ) * DIAG_LEN) ; 

for ( c=0 ; c<MAX_STUDENTS ; C++ ) 

name_list [ c ] . str = ( char *) malloc ( sizeof ( char ) * NAME_LEN ) ; 

/* allocate name list objects*/ 



assign_menu_attribs ( ) ; 
clear_student_space ( ) ; 



while (item < num_ts ) 

{ 

textattr ( rever se_color ) ; 
clrscr ( ) ; 
gotoxy ( 1,1); 

cpr intf ( "%s ", getcwd(buf, 



buf len ) ) ; 



program_root_dir = (char *) malloc ( sizeof (char) * ( strlen ( buf ) + 1 ) ); 

strcpy ( program_root_dir , buf); 

open_and_print_student_window ( &wh , num_ts ) ; 
item = point_and_shoot ( wh , textl, num_ts); 

WinDestroyWindow( wh ) ; 

switch ( item ) 

{. 

case -1: item = num_ts; 

break; 

case 1: strcpy ( st_name , add_or_edit_student ( ADD_STUDENT , "")); 
break; 

case 2: delete_or_edit_student ( DELETE_STUDENT ) ; 
break ; 

case 3: strcpy ( st_name , delete_or_edi t_s tudent ( EDIT_STUDENT ) ) ; 
break ; 

case 4: select_student ( " " ) ; 
break ; 




} /*switch*/ 

if ( strcmp ( st_name , "")) 

{ 

select_student ( st_name ) ; 





E 
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strcpy ( st_name , 

} / * i f * / 

free ( program_root_dir ) ; 
} /Awhile*/ 

WinDestroyWindowf wh ) ; 
WinTerminate ( ) ; 

f ree_all ( num_ts ) ; 
f lushall ( ) ; 

} 
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#inc 



#inc 

# inc 
#inc 

# inc 
#inc 

# inc 

# inc 

# inc 



lude< process . h> 
lude <string.h> 
lude < stdio . h > 
lude<alloc . h> 
lude<conio . h> 
lude<dir . h> 
lude < win . h> 
lude <pointsht.h> 
lude < keyboard. h> 
lude "student . h" 



#def ine REPORT "Report" 

#def ine STS "STS" 

#def ine SETUP "Setup" 

#def ine SPEECH _MGR "Speech Manager" 

#def ine EXIT "Exit" 

#def ine MAX_STUDENTS 20 /* max number of students*/ 

#de fine NAME_LEN 9 
#def ine AGE_LEN 3 
#def ine DIAG LEN 21 



extern 


short warning_color ; 


extern 


short win_color; 


extern 


short bdr_color; 


extern 


short EXPLODE; 


struct 


text__type *text; 


extern 


struct text_type *name 


struct 


student_type { 


char 


*name ; 


char 


*age ; 


char 


^diagnosis ; 


char 


*person_l ; 


char 


*person_2 ; 




} ; 



extern struct student_type student; 
extern char * program_root_di r ; 



void wtext_out ( HWND wh , int col, int row, char *str, int color); 



/*tttttttt*t**ttttt*********ttt***********tttttt*tt******t**tt***tt*/ 
/* FUNCTION open_and_pr int_main_menu %/ 

/* REMARKS opens and prints the main window of the student */ 

/ * program * , 

/* INPUT HWND *wh - pointer to the window handle %/ 

/* int num_ts - number of menu items */ 

/* OUTPUT 

/* CALLED BY main * / 

/* CALLS wtext_out %/ 

/ ***************t*****t*****tt****** ****** ttt t ttt tt tttt t ttt , 

void open_and_pr int_main_menu ( HWND *wh, int num_ts) 

{ 

int c ; 



O 
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*wh : WinCreateWindow( 7 , 15, 50, 12, win color , DBL_LIME_ALL_S IDES , bdr color, EXPLODE); 

WinCenterText ( *wh , 0, "Sound to Speech", win_color); 
for ( c=0 ; c<num_ts ; C++ ) 

wtext_out ( *wh , text [c] . col, text [c]. row, textfcl.str, win color); 



} 



/****************************ttt*tttttt%% xxxxxxxxxxxxxxxxxxxxxxxxxxx / 

/* FUNCTION assign_central_nenu_attnbs \j 



/* REMARKS 
/* 

/* INPUT 
/* OUTPUT 
/* CALLED BY 
/* CALLS 



assigns the menu attributes for the main menu to 
allow for point and shoot routine 



main 

al 1 oca te_text_ob ject 



*/ 

*/ 



*/ 

*/ 






void assign_central_menu_attr ibs ( void ) 

{ 

int lcol = 2 ; 
int first_row=2; 

0, STS, lcol, first_row); 

1, SETUP, lcol, f irst_row+2 ) ; 

2, SPEECH_MGR, lcol, f irst_row+4 ) ; 

3, REPORT, lcol, f i rst_row+6 ) ; 

4, EXIT, lcol, f irst_row+8 ) ; 

} 



allocate_text_ob.ject ( text , 
allocate_text_object ( text , 
al locate_text_ob j ect ( text , 
allocate_text_ob ject ( text , 
allocate_text_ob ject ( text , 



/ ’%%%%%%%%%%%% 
/* FUNCTION 
/% REMARKS 
/* 

/* INPUT 
/* OUTPUT 
/* CALLED BY 
/* CALLS 
/%%%%%%%%%%%% 



speech_mgr x / 
prompts the user to choose between Recognition * / 
functions and Recording Functions . % / 

main_menu x j 
allocate_text_object x / 



*/ 

*/ 



*/ 

*/ 



void speech_mgr ( void ) 

{ 



int k=0 ; 

struct text_type *text; 
HWND wh; 
int c = 0 ; 



#de f ine RECOGNITION "Recognition Functions" 
#def ine RECORDING "Recording Functions" 




text = (struct text_type* ) 
allocate_text_object ( text , 
allocate_text_object ( text , 
allocate_text_object ( text , 



malloc( sizeof ( struct text_type) 

0, RECOGNITION, 2, 2); 

1, RECORDING, 2, 4); 

2, EXIT, 2,7); 




* 3 ) ; 
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wh : WinCreateWindow( 10 , 27 , 26, 3, win color, DBL_LI8B ALLJIBES, bdr_co lor , EXPLODE); 

WinCenterText ( wh , 0, "Speech Manager", win_color); 
for ( c=0 ; c<=2 ; C++ ) 

wtext_out ( wh , text[cl.col, text[c] .row, textfcl.str, win_color); 



k = point_and_shoot ( wh , text, 3); 
WinDestroyWindow( wh ) ; 



for ( c=0 ; c< =2 ; C++ ) 
f ree ( text [c ] . str ) ; 
f ree ( text ) ; 
switch ( k ) 

{ 

case 1: spawnlp ( P_WAIT , "strain", 0); 
break ; 

case 2: spawnlp ( P_WAIT , "speech", 0); 
break ; 



} / * switch* / 



} 



/t********t******tttttt*t*tt******************tttt*t*tt*tt*******t.**/ 

/* FUNCTION main_menu * / 

/ * REMARKS main menu for student program. From here you can */ 

/* run STSMAIN, SETUP, SPEECH MANAGER, or return to the */ 

/* main system menu */ 

/* INPUT 
/* OUTPUT 

/* CALLED BY main */ 

/* CALLS assign_central_menu_attribs , open_and_jprint_main_menu */ 

/* speech_mgr */ 

/************************************* ******************************/ 



*/ 

*/ 



void main_menu ( void ) 

{ 

char buf [ 80 1 ; 
int . buf len=80 ; 

HWND wh; 

int item=0, num_ts = 5; 
extern short reverse_color ; 



text = (struct text_type*) malloc ( sizeof ( struct text type) * 5); 

assign_central_menu_attr ibs ( ) ; 
while (item < num_ts && item != -1 ) 

( 

textattr ( reverse_color ) ; 
clrscr ( ) ; 

gotoxy ( 1,1); 

cprintf("%s ", getcwd(buf, buflen)); 



ERiC 



open_and_print_main_menu ( &wh , num_ts ) ; 
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item = point_and_shoot ( wh , 
WinDestroyWindowf wh ) ; 



text , num_ts ) ; 



switch ( item ) 
{ 



case 


-1 : 


item = num_ts ; 
break ; 




case 


1 : 


spawnlp ( P_WAIT , 
break ; 


" stsmain' 


case 


2 : 


spawnlp ( P_WAIT , 
break ; 


"setup” , 


case 


3 : 


speech__mgr ( ) ; 
break ; 




case 


4 : 


spawnlp ( P__WAIT , 
break ; 


"report” , 


} / ^switch 


%/ 





} /*while*/ 

chdir ( program_root_dir ) ; 
for(item=0;item<=4;item++) 
free ( text [ item] . str ) ; 
f ree ( text ) ; 

} 



/ /*********** 
/* FUNCTION 
/* REMARKS 
/* INPUT 
/* OUTPUT 
/* CALLED BY 
/* CALLS 
/ /*********** 



*********** r* *******************************************/ 
title_screen ( ) */ 

explodes the title screen */ 



main * / 

get_key ( ) */ 

************************* # **^ 4 :# ^ # * 4 :#; * : **^ #4:; * ;;t:;t:;t:;t;;t:;t . ;t::t;;t;:t;;t;;t . :t/ 



void ti tle_screen ( void ) 

{ 

HWND wh ; 

char ver[5]="", buf [ 14 ]= "Version 
FILE *fp; 



*/ 

*/ 



wh : WinCreateWindowf 5 , 10, 60, 15, win_color, DBL_LINB_ALL_SIDBS , bdr color , EiPLODB); 
WinCenterText ( wh , 1, "Sound To Speech", win_color); 

fp = f open ( "version . dat" , "rt"); 

if (fp) 

f gets ( ver , 6 , f p ) ; 

fclose(fp) ; 
strcat(buf, ver); 



WinCenterText ( wh , 3, buf, win_color); 

WinCenterText ( wh , 5, "Copyright (C) ARC of the US, 1988-1991" 



WinCenterTextf wh , 8, 'Copyright (C) Genus Hicroprogramaing , Inc. 1988-1390', win_color); 

WinCenterText( wh , 10, "Copyright (C) Greenleaf Software, Inc. 1934-1983", win_color) ; 

WinCenterText (wh, 13, "Press a key to continue", win_color); 




win_color ) ; 
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get __key( ) ; 

WinDestroyWindowf wh ) ; 



} 
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main ( ) 

{ 



print f ( " \n\n%d\n\n" , 

} 



s i zeof ( char ) ) ; 
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ffdef ine DEBUG 



1 



ff include < coni o . h> 
ffinclude< string . h> 
ffinclude<alloc.h> 
#include<process.h> 
ff include < dir . h> 
rf include <stdio . h> 
ff include "win . h" 
ff include "point sht . h" 

# include "keyboard . h" 
ff include "student . h " 
ffdef ine BLANK STR LEN 8 " 



char *program_root_dir ; /% sts package root directory */ 

char name_buf [ 30 ] ; 

void clear_student_space ( void ) ; 

int put_message ( char *); 

void main_menu ( void ) ; 

int get_f ilenames ( char *, char *); 

void t i tle_screen ( void ) ; 



#def ine 
ff def ine 
#def ine 
#def ine 
w def ine 
#def ine 
#def ine 
#def ine 
# def ine 
ffdef ine 
ff def ine 
ffdef ine 
ffdef ine 



ADD_STUDENT 2 
SELECT_STUDENT 3 
DELETE_STUDENT 0 
EDIT_STUDENT 1 

MAX_STUDENTS 20 /* max number 

NAME_LEN 21 

AGE_LEN 3 

DIAG_LEN 24 

ADD_ST "Add Student" 

SELECT_ST "Select Student" 

DEL_ST "Delete Student" 

EDIT ST "Edit Student Description" 

EXIT ST "Exit" 



of students*/ 



short message_color = WHITE | REV_BLUE ! HI_INTENSITY ; 
short warning_color = WHITE ! REV_RED ! HI_INTENSITY ; 
short win_color = WHITE ! REV_GREEN ! H I_INTENS ITY ; 
short bdr_color = WHITE ! REV_GREEN ! HI_INTENSITY ; 
short reverse_color = WHITE | HI_INTENSITY ; 
short EXPLODE = TRUE; 

FILE *f open ( ) ; 



struct 


text__type *textl; 


struct 


text_type *name_list ; 


struct 


student_type { 


char 


♦name ; 


char 


♦age ; 


char 


♦diagnosis ; 


char 


♦personal ; 


char 


*person_2 ; 


char 


♦location ; 
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struct student_type student; 



/ ************************************ *t*tt***tttttt ************ 



/* FUNCTION 
/* REMARKS 
/* INPUT 
/* OUTPUT 
/* CALLED BY 
/* CALLS 



f ree_all 

frees all allocated memory space allocated by malloc 
int num_ts - number of main menu items 



main 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



/s**************************************************************.,..,..,..^ 

void free_all(int num_ts ) 

{ 

int c; 

f lushall ( ) ; 
fcloseall ( ) ; 
for ( c=0 ; c <num_ts ; C++ ) 
free(textl[c] ,str) ; 

for ( c = 0 ; c <MAX_STUDENTS ; C + + ) 
free ( name_list [c 1 .str) ; 
f ree ( student . name ) ; 
f ree ( student . age ) ; 
free ( student . diagnosis ) ; 
f ree ( student . person_l ) ; 
f ree ( student . per son_2 ) ; 
free ( student . location) ; 
f ree ( text 1 ) ; 
f ree ( name_li st ) ; 

} 



/* 


FUNCTION 


/* 


REMARKS 


/* 




/* 


INPUT 


/* 




/* 




/* 




/* 




/* 


OUTPUT 


/* 


CALLED BY 


/* 




/* 




/* 


CALLS 



wtext_out } 

Prints text to a window at a specified row, col and 
color 

1IWND wh - window handle to print to 

int col - column of window to print at 

int row - row of window to print at 

char *str - string to print 
int color - color attributes for the string to print */ 

*/ 

°pen_and_pr int_student_window , display_st_inf o_str , */ 

create_add_student_window, add_or_edi t_student , */ 

delete_or_edit_student 

*/ 



*/ 

%/ 

*/ 

*/ 

*/ 

*/ 

*/ 



void wtext_out ( HWND wh , int col, int row, char *str, int color) 

{ 

WinSetCursorPos ( wh , row, col); 

WinTextOut ( wh , str, color); 

} 

/s************************************************^^^^^.,,.^,^,^,^ 

ERIC 



11/07/1991 



VSTUDENT . C 



Page 3 of 20 



/* FUNCTION put_warning_msg %/ 
/* REMARKS Opens a red and white window at the center of the * / 
/* screen, and prints a message %/ 
/* INPUT char *str - string or message to print */ 
/* OUTPUT */ 
/* CALLED BY save_s tudent_inf o %/ 
/ * CALLS get_key */ 



void put_warning_msg ( char *str) 

{ 

HWND wh; 

wh = WinCreateWindow( 11, 10, 60, 3, warning_color, DBL_LINE_SIDES, warning_color, EXPLODE) ; 
WinCenterText ( wh , 1, str, warning_color ) ; 

get_key ( ) ; 

WinDestroyWindow ( wh ) ; 

} 






/* 


FUNCTION 


/* 


REMARKS 


/* 




/* 


INPUT 


/* 


OUTPUT 


/* 


CALLED BY 


/* 


CALLS 



ass ign_menu_att ribs 

assigns the menu attributes for the main menu to 
allow for point and shoot routine 



main 



allocate_text_ob ject 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



/**************************************U******:tU********************/ 



void assign_menu_attr ibs ( void) 

{ 

int first_row = 2; 
int lcol = 2 ; 

allocate_text_object ( textl , 0, 
allocate_text_object ( text 1 , 1, 

allocate_text_object ( textl , 2, 
allocate_text_object ( textl , 3, 
allocate_text_ob ject ( text 1 , 4, 



ADD_ST , lcol, first_row); 
DEL_ST , lcol, f irst_row+2 ) ; 
EDIT_ST, lcol, f irs t_row+4 ) ; 
SELECT_ST , lcol, f irst_row+6 ) ; 
EXIT_ST, lcol, f i rst_row+ 1 0 ) ; 



} 






/* 

/* 

/* 

/* 

/* 

/* 

/* 

/* 



FUNCTION 

REMARKS 

INPUT 

OUTPUT 
CALLED BY 
CALLS 



open_and_print_student_window 
opens and prints the main window of the student 
program 

HWND *wh - pointer to the window handle 
int num_ts - number of menu items 

main 

wtext out 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



/s************************************************************^^^^ 



void open_and_pr int_student_window ( HWND *wh, int num_ts) 
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twh : WinCreateWindow( 7 f 15, 50, 14, win_color, DBL_LI!1B_ALL_SIDES, bdr_color, EXPLODE); 

WinCenterText ( *wh , 0, "Student"', win_color); 
for ( c=0 ; c <num_ts ; C++ ) 

wtext_out ( twh , textile] .col , text 1 [ c ]. row , text 1 [ c ] . s tr , win_color ) ; 



/* 


FUNCTION 


/* 


REMARKS 


/* 


INPUT 


/* 




/* 




/* 


OUTPUT 


/* 


CALLED BY 


/* 


CALLS 



create_add_student_window 



HWND tst_wh - pointer to the student window 
int lcol - left column the attribute names will be 
printed at 

add_or_edi t_student 
wtext out 



*/ 



creates and prints the textl for the attribute input */ 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



void create_add_student_window( int OPERATION, HWND *st_wh, int lcol) 



*st_wh = WinCreateWindow( 7 , 15, 50, 
swi tch ( OPERATION ) 

{ 



18, win_color, DBL_LI!JB_S IDES , bdr_color, EXPLODE); 



case ADD_STUDENT : WinCenterText ( *s t_wh , 0, 

break ; 

case EDIT_STUDENT : WinCenterText ( *st_wh , 0, 

break ; 

} / *swi tch* / 



Add Student " , 
Edit Student" 



win_color ) ; 
win_color ) ; 



wtext_out ( *st_wh , 
wtext_out ( *st_wh , 
wtext_out ( *st_wh , 
wtext_out ( *st_wh , 
wtext^out ( *st_wh , 
wtext_out ( *st_wh , 
wtext_out ( *st_wh , 
wtext_out ( *st_wh , 
wtext^out ( *st_wh , 
wtext_out ( *st_wh , 
wtext_out( *st_wh, 
wtext_out ( *st_wh , 
wtext_out( *st_wh, 
wtext_out ( *st_wh , 



lcol , 2 , "Nam£ : [ " , win_color ) ; 

lcol+5+NAME_LEN , 2, " ] " , win_color); 
lcol, 4, "Aged" i win_color); 
lcol+4+AGE_LEN , 4, win_color); 

lcol, 6, "Diagnosis :[" , win_color); 
lcol+10+DIAG_LEN , 6, "]", win_color ) ; 
lcol, 8, "Person 1:[ M , win_color); 
lcol +9 +NAME_LEN , 8, " ] " , win_color); 
lcol, 10, "Person 2 : [ " , win_color); 
lcol+9+NAME_LEN , 10, "]", win_color); 

lcol, 12, "Location :[", win_color); 
lc o 1 + D I AG_LEN+ 10, 12, " ] M , wi n_color ) ; 

lcol, 14, Exit and Save”, w in_col or ) ; 
lcol, 16, "Select Student", win_color); 



/* 

clear_student_space( ) ; 



■clear out new space 



■*/ 



} 



O 

ERIC 



/* FUNCTION display_st info str 



. */ 
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/* REMARKS displays the individual attributes of each student */ 



/* INPUT HWND wh - window handle to print to */ 
/* char *str - character string to print */ 
/* char *blank - blank string */ 
/* int col - column to print at */ 
/* int row - row to print at %/ 
/* OUTPUT */ 
/* CALLED BY display_student_info */ 
/* CALLS wtext_out */ 



/ ***************************** ***************************** tttttt***/ 



void display_st_info_str (HWND wh , char *str, char *blank , int col, int row) 

{ 

if (strcmplstr, NULL) && strcmp(str, "-")) 
wtext_out ( wh , col, row, str, win_color); 
else 

wtext_out ( wh , col, row, blank, win_color); 



} 



/% 


FUNCTION 


di splay_student_inf o 


*/ 


/* 


REMARKS 


center for printing student attributes 


*/ 


/% 


INPUT 


HWND wh - window to print to 


*/ 


/* 




int lcol - column to print at 


*/ 


/* 


OUTPUT 




*/ 


/* 


CALLED BY 


add__or_edi t_student 


*/ 


/* 


CALLS 


di splay_s t_inf o_str 


*/ 



/******************************************************************.,./ 



void display_student_info ( HWND wh , int lcol) 



display_st_info_str ( wh , 
display_st_inf o_str ( wh , 
display_st_inf o_str ( wh , 
display_st_inf o_str ( wh , 
display_st_inf o_str ( wh , 
display_st_inf o_str ( wh , 



student .’name , BLANK_STR_LEN_8 , lcol + 6, 2); 
student. age, " ", lcol+5, 4); 

student .diagnosis , BLANK_STR_LEN_8 , lcol+11, 6); 
student . personal , BLANK_STR_LEN_8 , lcol+10, 8); 
student . person_2 , BLANK_STR_LEN_8 , lcol+10, 10); 
student . location , " ", lcol+11, 12); 



wtext_out ( wh , lcol, 14, "Exit and Save", win_color); 
wtext_out ( wh , lcol, 16, "Select Student", win_color); 



} 

/s***************************************************************^^ 
/* FUNCTION write_str2f ile */ 
/* REMARKS If the string exists then the string is written to */ 



/* a file, else a is written. %/ 
/ * INPUT FILE *fp - file to write to %/ 
/* char *str - string to write */ 
/* OUTPUT */ 
/* CALLED BY update_student_list_f ile , save_student info */ 
/* CALLS * / 



11/07/1991 



VSTUDENT . C 



Page 6 of 20 



/t*t***tttttttttttttt***t**********tttttt******tt*tttt*************t/ 

void write_str2f ile ( FILE *fp, char *str) 



if ( str && strlen ( str ) > 0 ) 

{ 

fprintf(fp, "%s\n", str); 

) /*if */ 
else 
{ 

fprintf(fp, "-\n" ) ; 

} /*else*/ 

) 

/*******************************************************************/ 
/* FUNCTION update_student_list_f ile */ 

/* REMARKS updates the student list by adding the new name */ 

/* INPUT FILE *fp - file handle for the student list file */ 

/* OUTPUT */ 

/* CALLED BY save_student_info */ 



/t*tt*t***t*****t*ttttt*tt*t****tt*tt***ttttttttttttt*ttt*tttttt%*t*/ 



void update_student_list_f ile ( void) 

{ 

FILE *fpl ; 

char str[ 10]="" ; 

int FOUND = FALSE; 

/* check to ensure that the name does not exist in the file */ 
fpl = f open ( "studlist . dat " , "rt"); 

if (fpl) 

{ , 

while (! feof ( fpl ) && strcmp(3tr, "")) 



fgets(str, 10, fpl); 
str[strlen(str)-l]=0; 
i f ( ! strcmp ( str , student . name ) ) 
FOUND = TRUE; 

} / * wh i 1 e * / 

} /*if */ 
f flush ( fpl ) ; 
fclose ( fpl ) ; 



if ( ! FOUND ) 

{ 

fpl = f open ( "studlist . dat " , "at"); 

if (fpl) 

( 

wr i te_str2f i le ( f pi , student . name ) ; 



/* CALLS 



write str2file 



} /*if */ 
fflush( fpl ) ; 
fclose ( fpl ) ; 
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}/*if*/ 



} 

/*******************************************************************/ 



/% FUNCTION save_student_inf o %/ 
/* REMARKS saves the student attributes to a file */ 
/* INPUT */ 
/* OUTPUT */ 
/* CALLED BY */ 

/* CALLS put_war inng_msg , write_str2f ile */ 



/S************************************************************;,;***;,.*/ 



void save_student_inf o ( int OPERATION) 

{ 

FILE *fp; 
char buf [100] ; 
char buf 1 [75] ; 
char dir_name[9]; 
char ins_dash[2]; 
char *blnk_pt; 

char fst_prt[5], lst_prt[5]; 




strcpy(buf, program_root_dir ) ; 
strncpy ( dir_name , student . name , 8 ) ; 
blnk_pt = strstr ( dir_name , " "); 
ins_dash[0] 
if (blnk_pt != NULL) 

{ 

strncpy( fst_prt , strtok ( dir_name , " " ) , 4 ) ; 
strncpy ( lst_prt , strtok ( NULL , " " ) , 4 ) ; 
strncpy ( dir_name , f st_prt , 4 ) ; 
strncat ( dir_name , ins_dash , 1 ) ; 
strncat ( dir_name , lst_prt , 4 ) ; 

} 

if ( mkdir ( dir_name ) && OPERATION == ADD_STUDENT ) 
put_warning_msg ( " You need a Unique name!"); 
else 



{ 

strcat(buf, "\\"); 
strcat(buf, dir_name); 
chdir ( buf ) ; 

fp = f open ( "students . dat" , "wt" ) ; 
if (fp) 

{ 

write_str2f ile ( fp , student . nameV) ; 
write_str2f ile ( f p , student . age ) ; 
wr i te_str2f ile ( f p , student .diagnosis) ; 
wr i te_str 2f ile ( f p , student . person_l ) ; 
wr i te_str 2f ile ( f p , student . person_2 ) ; 
wr i te_str 2 f i le ( f p , student . location ) ; 

} /*if */ 
f flush ( f p ) ; 
fclose ( fp ) ; 

chdir ( program_root_dir ) ; 
if (OPERATION == ADD_STUDENT ) 
update_student_list_f ile ( ) ; 

}/*else*/ ft A 
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} 



/tttttttttttttt***************t*********tttt**********tttt*ttttttttt/ 



/% FUNCTION clear_student_space */ 
/* REMARKS sets all student attributes to null */ 
/* INPUT */ 
/* OUTPUT */ 
/ * CALLED BY creat e_add_student_window , add_or_edi t_s tudent */ 
/* CALLS */ 



/*******************************************************************/ 

void clear_student_space ( void ) 

{ 

strcpy ( student . name , 
strcpy ( student . age , 
strcpy ( student . diagnosis , 
strcpy ( student . person_l , 
strcpy ( student . person_2 , 
strcpy ( student . location , 

} 

/S***********************************************************-!.*.!..,..!..,.*/ 



/* FUNCTION read_s t r_f rom_f i le */ 
/ * REMARKS reads a string from a file */ 
/* INPUT FILE *fp - file to read from */ 
/* char *str - pointer to string to return %/ 
/* OUTPUT */ 
/* CALLED BY read_student_data */ 
/* CALLS */ 



void read_str_from_f ile (FILE *fp, char *str) 

< » 

f gets ( str , NAME_LEN+ 1 , fp); ? 

str [ strlen ( str ) - 1 ] = 0; 



/%%%%%%%%%%%%%%%%%%%%%% / 



/* FUNCTION 
/* REMARKS 
/* INPUT 
/* OUTPUT 
/* CALLED BY 
/* CALLS 



check4null 
checks to see if the string is blank 
char *str - string to check 

read student data 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



/**************************************************************£;,.£££/ 



void check4null ( char *str) 

{ 

if ( ! strcmp ( str , 
strcpy(str, NULL); 

} 



/%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% / 
/% FUNCTION read student data */ 
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/* 

/* 

/* 

/* 

/* 



REMARKS 

INPUT 

OUTPUT 

CALLED 

CALLS 



BY 



/*********** 



reads in the student data from the file */ 

char *name - the students name/directory to read from*/ 

* / 

add_or_edit_student */ 

read_str_f rom_f ile , read_student_data */ 

********************************* ttt*************t****t*/ 



void read_student_data ( char *name) 

{ 

FILE *fp; 

char str [NAMEJLEN1 ="" , buf[100]; 
name = &name_buf [ 0 ] ; 
strcpy(buf, program_root_dir ) ; 
strcat(buf, "\\"); 
strcat(buf, name); 



chdir ( buf ) ; 

fp = fopen( "students .dat" , "rt"); 



read_str_f rom_f ile ( fp , str); 
strcpy( student . name , str); 
check4null ( student . name ) ; 



read_str_f rom_f ile ( fp , str); 
strcpy ( student . age , str); 
check4null ( student . age ) ; 



read_str_f rom_f ile ( fp , str); 
strcpy ( student . diagnosis , str); 
check4null ( student . diagnosis ) ; 



read_str_f rom_f ile ( fp , str); 
strcpy ( student . person_l , str); 
check4null ( student . person_l ) ; 

read_str_f rom_f ile ( fp, str); 
strcpy ( student . per son_2 , str); 
check4null ( student . person_2 ) ; 

read_str_f rom_f ile ( fp , str); 
s trcpy ( student . locat ion , str); 
check4null ( student . location ) ; 



fflush( fp) ; 
f close ( fp ) ; 

chdir ( program^root_dir ) ; 

} 



/ *********** 

/* FUNCTION 
/* REMARKS 
/* INPUT 
/* 



O 



/* 

/* 

/* 



ERIC 



OUTPUT 
CALLED BY 
CALLS 



************************************************ 

add_or_edit_student 

control center for adding a student to the STS 

int OPERATION - ADD or EDIT 

char *name - name of student to edit 

main 

create_add_student_window , clear_student_space 



******** / 
*/ 

system*/ 

*/ 

*/ 

*/ 

*/ 

, */ 
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/* display_student_inf o , wtext_out, read_student_da ta */ 

/****************tt*ttt**t***t*tttt**t******ttttttt*tt****ttt*ttt*tt/ 

char* add_or_edit_student( int OPERATION, char *name ) 

{ 

HWND st_wh ; 

int lcol = 2, num_items = 7; 

char *str; 

int item = 0; 

int FINISHED = FALSE; 

int last_key =0; 

int rc = 0 ; 

str = (char * ) malloc ( sizeof ( char ) * NAME_LEN) ; 

create_add_student_window ( OPERATION , &st_wh, lcol); 
switch (OPERATION) 

{ 

case ADD_STUDENT: clear_student_space ( ) ; 

break ; 

case EDIT_STUDENT : read_student_data ( name ) ; 

break; 



» 



} / *swi tch* / 
textattr ( win_color ) ; 
while ( ! FINISHED) 

{ 

di splay_student_inf o ( st_wh , lcol) ; 
switch ( item ) 

{ 

case 0 : if (OPERATION != EDIT_STUDENT ) 

{ 



&last_key ) ) ; 



strcpy (str, get_str ( st_wh , NAME_LEN- 1 , 

if (strcmp(str, NULL)) 

strcpy ( student . name , str); 

break ; 

} /*els e*/ 
else 



i tem= 1 ; 



lcol +6 , 




case 1 : strcpy (str, get_str ( st_wh , AGE_LEN- 1 , lcol + 5, 4, &last key) ) • 

if (strcmplstr, NULL)) ' 

strcpy ( student . age , str); 
break ; 

case 2 : strcpy(str, get_str ( st_wh , DIAG_LEN- 1 , lcol+11, 6, &last key))* 

if (strcmp(str, NULL)) — 

strcpy ( student . diagnosis , str); 
break ; 

case 3 : strcpy(str, get_str ( st_wh , NAME_LEN- 1 , lcol+10, 8, &last key))* 
if (strcmp(str, NULL)) — 

strcpy ( student . person_l , str); 
break ; 

case 4 : strcpy( str , get_str(st_wh , NAHB_LBN- 1 , lcol+10, 10, ilastjrey)); 
if (strcmpfstr, NULL)) 

strcpy ( student . person_2 , str); 
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break ; 

case 5 : strcpy ( str, get_=tr { st_wh , DIAGJ.EU-1, lcol+11, 12, llast_key|); 
if (strcraplstr, NULL)) 

strcpy ( student . location , str); 
break; 

case 6 : wtext_out ( st_wh , lcol, 14, "Exit and Save", reverse_color ) ; 

last_key =0; 

while ( last_key != DOWN 11 lastkey != RETURN 11 last_key !: ESC 11 last_key !: UP) 
last_key = get_key( ) ; 



if ( last_key == ESC jj last_key = = RETURN) 

FINISHED = TRUE; 
break ; 

case 7 : wtext_out ( st_wh , lcol, 16, "Select Student", reverse_color ) ; 
last_key = 0; 

while (last_key != RETURN && last_key != ESC && last_key != UP) 
last_key = get_key( ) ; 



if ( last_key == ESC jj last_key == RETURN) 
FINISHED. = TRUE; 
if (last_key == RETURN) 
rc = 1 ; 
break ; 



} / *switch*/ 



switch ( las t_key ) 

{ 



case UP: if (--item <0) 

item = 0 ; 
break ; 

case RETURN: if (++item > num_items) 
item = num_i terns; 
break ; •, 

case DOWN: if (++item > num_items) 

item = num_items; 
break ; 



case ESC: FINISHED = TRUE; 



break ; 

} /* switch*/ 



} /*while*/ 

WinDestroyWindow ( st_wh ) ; 
if ( last_key == RETURN) 

save_student_inf o ( OPERATION ) ; 

free ( str ) ; 

i f ( rc ) 

return student . name ; 
else 

• -r^* "t - i ] "T* * * M * 

BEST COPY AVAILABLE 




} 
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/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/ 
/* FUNCTION delete_file %/ 

if the file or wildcard exists, then the file is */ 

deleted without any screen output */ 

char *file - file to delete %/ 

the success of the deletion * / 

delete_data_f iles %/ 



/* REMARKS 
/ * 

/* INPUT 
/* OUTPUT 
/* CALLED BY 



/* CALLS %/ 

/************ t****t*ttt ********* ****x************x**********x******t/ 



int delete_f ile ( char *file) 

{ 

char del_cmd [ ] = "del " ; 

char command[19]; 
struct ffblk f; 
int a=0 ; 

if ( f indf irst ( f ile , &f , a)==0) 

{ 

strcpy( command , del_cmd); 
strcat ( command , file); 
return system ( command ) ; 

} /*if */ 
else 

return - 1 ; 



/ %%%%%%%%%%%%%%%%%%%%%% XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/ 



/* 


FUNCTION 


del ete_data_f iles 


X / 


/* 


REMARKS 


deletes a file from a path; handles wildcards 


X / 


/* 


INPUT 


char *path - path to delete 


*/ 


/* 




int color - color to display the message 


X/ 


/* 


OUTPUT 


int - success or not 


X/ 


/* 


CALLED BY 


delete__student_named 


X/ 


/* 


CALLS 




X/ 



/ XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX / 



int deleto_data_f iles ( char *path, int color) 

{ 

int x = 0, c; 

char str[100]=""; 

extern char directory [ 50 ][ 1 4 1 ; 

strcpy ( str , program_root_dir ) ; 
strcat(str, "\\"); 
strcat(str, path); 

chdir ( str ) ; 

c = get_f ilenames ( "c : " , "*.dat"); /* the address is blown 

when this function returns */ 

c- = 2 ; 

textattr ( color ) ; 
while ( x < = c ) 

{ 

strcpy(str, directory [x 1 ) ; 
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strcat ( str , ".*"); 

delete_f ile ( str ) ; 

x + + ; 

} /*while*/ 

delet e_f i le ( " * . da t " ) ; 
delete_f ile ( " * . vpc" ) ; 
delete_f ile ( "null . *" ) ; 

for ( x=0 ;x<=9 ;x++ ) 

{ 

sprintf(str, "*.%ld*", x); 
delete_f ile ( str ) ; 

} /*for*/ 

delete_file( "num . " ) ; 
chdir ( program_root_dir ) ; 

puts ( "Returning c" ) ; 
return c; 

} 

/**********************************ttt*t**t ************* ************/ 



/* FUNCTION delete_name_f rom_student_list */ 
/* REMARKS deletes a student name from the studlist.dat file */ 
/ * INPUT char *name - name of student to delete */ 
/ * OUTPUT * / 
/ * CALLED BY delete_student_named */ 
/* CALLS */ 



void delete_name_f rom_student_list ( char *name) 

{ 

FILE *fpl, *fp2 ; 

char *sname; /* student name read in from the file*/ 

fpl = fopen ( "studlist . dat" , "rt”;); 

if (fpl) 

{ 

sname = (char *) malloc ( sizeof ( char ) * 9); 

strcpy ( sname , "qwerty\n" ) ; 
fp2 = f open (" temp . dat " , "wt"); 

fgets(sname, 9, fpl); 

while (! feof ( fpl ) && s trcmp ( sname , NULL)) 

{ 

sname [ strlen ( sname )- 1 ] = 0; 
if ( strcmp ( sname , name)) 

{ 

fputs( sname, fp2); 
fputc ( ’ \n ’ , f p2 ) ; 

} /*if */ 

strcpy ( sname , NULL); 
fgets(sname, 9, fpl); 

}/* while*/ 
f flush ( fp2 ) ; 
fclose ( fp2 ) ; 
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remove ( 
rename ( 
} /*if */ 



"studlist.dat") ; 
"temp.dat”, "studlist.dat" 



f flush ( fpl ) ; 
fclose ( fpl ) ; 

} 






/* FUNCTION delete_student_named %/ 

/* REMARKS yes/no query to verify deletion of user */ 

/* INPUT char *name - name of user to delete */ 

/* OUTPUT */ 

/* CALLED BY delete_or_edit_student %/ 

/* CALLS delete_data_f i les , delete_name_f rom_student_list */ 



/**t*t*ttttt*t******t**tttttttttttt****t******tt**tttttt***tt**ttttt/ 

void delete_student_named ( char *name) 

{ 

HWND w_wh; 
char *buf ; 

short DELETED = FALSE; 



all of %s ' s files 



buf = (char *) malloc ( si zeof ( char ) * 50); 

w_wh : WinCreateWindowfll, 10, 60, 5, warning_color, DBL_LIKE_SIDBS . warning color , EXPLODE'; ; 
sprintf(buf, "Deleting %s will erase 
WinCenterText ( w_wh , 1, buf, warning_color ) ; 

WinSetCursorPos ( w_wh , 3, 19); 

if ( delete_data_f iles ( name , warning_color ) > -1) 
DELETED = TRUE; 
else 

DELETED = FALSE; 



name , name ) 



printf ( "Returned from delete data files"); 
get_key( ) ; *• 



WinDestroyWindow( w_wh ) ; 
if (DELETED) 

{ 

rmdir ( name ) ; 

/* delete name from studlist.dat file */ 
delete_name_f rom_student_list ( name ) ; 

}/*if*/ 



f ree ( buf ) ; 

) 




/* FUNCTION delete_or_edit_student 

/* REMARKS command center for deleting or editing a student 

/* INPUT int OPERATION - DELETE or EDIT a student 

/* OUTPUT 

/* CALLED BY main 

/* CALLS wtext_out , delete_student_named 

/**t**t*****ttt**ttt**t****t*t****ttttttt******ttttt*ttttttt**t**** 
char* delete_or_edit_student ( int OPERATION) 

121 



* 

* 

* 

* 

* 

* 

* 

* 



/ 

/ 

/ 

/ 

/ 

/ 

/ 

/ 
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FILE 
char 
int c 
HWND 



HWN'D 

char 



*fp; 
*str ; 



» x , 

sl_wh ; /^student list window handle*/ 

w_wh ; /* warning window handle*/ 

st_name [NAME_LEN+ 1 1 = " " ; 



str = (char * ) malloc ( si zeof ( char ) * NAME_LEN ) ; 



fp = fopen ( "studlist . dat” , "rt"); 
if (fp) 

{ 

c = 0 ; 

while( ! feof( fp) && c < MAX_STUDENTS ) 

{ 

f gets (name_list[c] .str, NAME_LEN+1 , fp); 
if ( name_list [c ] . str ) 

{ 

name_l ist T c ] . str [ strlen ( name_list [c ] .str)-ll=0; 

C + + ; 

} /*for*/ 

} /* while * / 
fflush( fp) ; 
fclose ( fp) ; 




x 

c- 

if 



c - 1 ; 

2 ; 

( 0 = 0 ) 



{ 

sl_wh : IfinCreateWindowf 1 , 1, 25, x+2 , win_color, DBL_LI!!E SIDES, bdr color , EXPLODE); 

WinCenterText ( sl_wh , 0, "Student List", win_color); 
while ( c > = 0 ) 

{ » 

w t e x t o ut ( s 1 w h , 2, c+1, name_list [ c ] . str , win_color) 

namej i st [ c ] . col = 2; 
name__l i st [ c ] . row = c+1; 
c ; 



} /Awhile*/ 
} /*if*/ 



H 




if ( x > 0 ) 

{ 

wh : WinCreateWindow( 11 , 10, 60, 3, niessage_color, DBLLINE SIDES, aes 3 age_color, EXPLODE) ; 
switch (OPERATION) 

{ 

case DELETE_STUDENT : WinCenterText ( w_wh , 1, 

’’Point to the student to delete. 
^ break; 

case EDIT_STUDENT : WinCenterText ( w_wh , 1, 

’’Point to the student to edit.”, 
break ; 

) /*swi tch* / 




if <get_key< ) ! = ESC) 

{ 




, message_color ) 
message_color ) ; 
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WinDestroyWindow ( w_wh ) ; 
c = point_and_shoot ( sl_wh , 
if ( c > — 1 ) 
switch (OPERATION) 



{ 



name_list , x ) ; 



case DELETE_STUDENT : delete _student named ( name _1 i st T c- 1 1 . str ) ; 

break; 

case EDIT_STUDENT : strcpy ( st_saae . add_or_edit_student(EDIT_8TUDEST, nase_list[c-l).str!!: 

break ; 

} / *switch*/ 

} /*if */ 
else 

WinDestroyWindow ( w_wh ) ; 



} /*if */ 
else 
{ 

put_message ( "No students found!"); 
}/*else*/ 



} /*if */ 
else 

put_message( "No students found!"); 
f close ( f p ) ; 
f ree ( str ) ; 

WinDestroyWindow( sl_wh ) ; 
if ( strcmp ( st_name , "")) 

return st_name; 
else 

return " " ; 






/* FUNCTION 
/* REMARKS 
/* INPUT 
/* OUTPUT 
/* CALLED BY 
/* CALLS 



put_message 
puts a message in a window 
char *str - message 

whether the user wishes to continue or ESC 
get_key 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



ERiC 



int put_message ( char *str) 

{ 

HWND vh; 
int key; 

wh = WinCreateWindow( 1 1 , 10, 60, 3, aessage_color, DBLJJNBJIDES, aessage color , EXPLODE); 
WinCenterText ( wh , 1, str, message_color) ; 

key = get_key< ) ; 

WinDes troyWindow{ wh ) ; 
if (key == ESC) 
return - 1 ; 
else 

return 1 ; ij 
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/************************************ *****t***na** ******************* / 



/* 


FUNCTION 


display_student_list 


*/ 


/* 


REMARKS 


opens a window and displays a student list for 


*/ 


/* 




point and shoot 


*/ 


/* 


INPUT 




*/ 


/* 


OUTPUT 


int - student id of the selected student 


*/ 


/* 


CALLED BY 


select_student 


*/ 


/* 


CALLS 


point_and_shoot , put_message 


* / 



/*******************************************************************/ 

int display_student_li st ( void ) 

{ 

char *str; 
int c , x ; 

FILE *fp ; 

HWND sl_wh; 

str = (char * ) malloc ( sizeof ( char ) * 10); 
fp = fopen ( "studlist . dat" , "rt" ) ; 
if (fp) 

{ 

c = 0 ; 

while( ! feof ( fp) && c < MAX_STUDENTS ) 

{ 

fgets ( name_list [c ] . str , NAME_LEN , fp); 
if ( name_l ist [ c ] . str ) 

{ 

name_list [c] .str[strlen( name_list [c] . str ) -1 1 =0 ; 

C + + ; 

}/*for*/ 

} /*while*/ 



er|c 



X = c - 1 ; i 

c- = 2 ; 
if (0=0) 

{ 

sl_wh : WinCreateWindow( 1, t, 25, x+2, wincolor, D8L_LI"E_S IDES , bdr_color, EXPLODB) ; 

WinCenterText ( sl_wh , 0, "Student List", win_color); 
while ( c > = 0 ) 

{ 

wtext_out ( sl_wh , 2, c+1, name_list Tc ] . str , win_color); 
name_li s t [ c ] . col = 2; 
name_list [c ) . row = c+1; 
c-- ; 

} / *whi le * / 

}/*if*/ 

if ( x > 0 && put_message ( "Point to the desired student.") > -1 ) 

c = point_and_shoot ( sl_wh , name_list, x); 
else 

if (x <= 0) 

put_message( "No students found!"); 

) /*if */ 
else 
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{ 

put_message( "No students found!"); 
c = -1 ; 

} /*else*/ 



f close ( f p ) ; 
f ree (str) ; 

WinDestroyWindow ( sl_wh ) ; 
return c ; 

} 



/ *************************************************** / 
/* FUNCTION select_student */ 

/* REMARKS prompts the user to select the student */ 

/* INPUT char *name - student’s name */ 

/* OUTPUT . -- t/ 

/* CALLED BY main */ 

/ * CALLS display_student_list */ 

/*************************************************************** tttt/ 



void select_student ( char *name) 

{ 

int c ; 

char buf [100] ; 




if 

{ 



( ! s t rcmp ( name , 



" " ) ) 



c = display_student_l ist ( ) ; 
if ( c > -1) 

{ 

strcpy ( buf , prograra_root_dir ) ; 
strcat(buf, "\\"); 
strcat(buf , name_list [c-1 ] .str) ; 
chdir ( buf ) ; ! 

main_menu ( ) ; 

textattr ( reverse_color ) ; 
clrscr ( ) ; 

} /*if */ 

} /*if */ 
else 



{ 

strcpy(buf , program_root_dir ) ; 
strcat(buf, "\\"); 
strcat(buf, name_list [c-1 ] . str ) ; 
chdir ( buf ) ; 



main_menu ( ) ; 

textattr ( reverse_color ) ; 
clrscr ( ) ; 

} / * e 1 s e * / 




} 



void main(void) 
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int 

int 

char 



item = 0 ; 
c ; 

st_name [NAME_LEN+1 1 = 



HWND wh; 



int num_ts =5; /* num 

#define buflen 80 
char buf [buflen]; 



tt »» . 



items 



on main menu*/ 



#if ! DEBUG 

spawnlp ( P_WAIT , "bird" , 0) ; 
#endif 



puts( “Trying to ini tialize” ) ; 
Winlnitialize ( ) ; 
puts { “Initialized" ) ; 



textattr ( reverse_color ) ; 

clrscr { ) ; 

ti tle_screen ( ) ; 

puts ( "Title Screen" ) ; 

naae_list = (struct text_type*) malloc( sizeof( struct text_type) t HAXJTUDBBTS); 

t ext 1 = (struct text_type*) malloc ( sizeof ( struct text_type) * num_ts > 
student. name = (char *) malloc { sizeof ( char ) * NAME_LEN) ; 
student. age = (char *) malloc { s i zeof ( char ) * AGE_LEN ) ; 

student . diagnosis = (char *) malloc ( sizeof ( char j * DIAG_LEN) ; 
student .personj = (char *) malloc ( s i zeof ( char ) * NAME_LEN ) ; 
student . person_2 = (char *) malloc (sizeof (char) * NAME_LEN ) ; 
student . location = (char *) malloc ( si zeof ( char ) * DIAG_LEN); 



for ( c = 0 ; c<MAX_STUDENTS ; c + + ) 

name_list [ c ] . str = ( char* ) mall oc( sizeof ( char ) 
/* allocate name list objects*/ 



* NAME LEN) 



assign_menu_attr ibs ( ) ; * 

clear_student_space( ) ; ?• 

while (item < num_ts ) 

{ 

textattr (reverse_color) ; 
clrscr ( ) ; 
gotoxy ( 1,1); 

cprintf(“%s ", getcwd(buf, buflen)); 



program_root_dir = (char *) malloc ( sizeof (char) * ( s tr len ( buf ) + 1 ) ); 

strcpy ( program_root_dir , buf); 

°pen_and_pr int_s tudent_window ( &wh , num_ts) ; 
item = point_and_shoot ( wh , textl , num_ts ) ; 

WinDestroyWindow ( wh ) ; 




switch ( item ) 

{ 

case -1: item = num_ts; 

break ; 

case 1: s trcpy ( s t_name , add_or__edi t_s tudent ( ADD_STUDENT , "" ) ) ; 
break ; 
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case 2: delete_or_edi t_student ( DELETE JSTUDENT ) ; 
break ; 

case 3: strcpy ( st_name , delete_or_edi t_student ( EDIT_STUDENT ) ) ; 
break ; 

case 4: select_student ( " " ) ; 
break ; 

} /*switch*/ 

if ( strcmp ( st_name , "")) 

{ 

select_student ( st_name ) ; 
strcpy ( st_name , 

} /*if */ 

f ree ( program_root_dir ) ; 

} /* while*/ 

WinDestroyWindow( wh ) ; 

WinTerminate ( ) ; 

free_all(num_ts) ; 
f lushall ( ) ; 

} 





ERIC 
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5? me 
#inc 
#inc 



#inc 

vine 



vine 



lude< process . h> 
lude < string . h> 
lude<stdio . h> 
lude<alloc . h> 
lude < conio . h > 
lude <dir . h> 
lude < win .h> 
lude<pointsht . h> 
lude < keyboard . h> 
lude " student . h" 



#def ine REPORT "Report" 
vdef ine STS "STS" 

#def ine SETUP "Setup" 

vdefine SPEECH_MGR "Speech Manager" 

#def ine EXIT "Exit" 

vdefine MAX_STUDENTS 20 /* max number of students*/ 

#def ine NAME_LEN 9 
vdefine AGE_LEN 3 
vdefine DIAG LEN 21 



extern 


short warn ing_co lor ; 


extern 


short win__color; 


extern 


short bdr_color; 


extern 


short EXPLODE; 


struct 


text_type *text; 


extern 


struct text_type *name 


struct 


student_type { 


char 


*name; 


char 


*age ; 


char 


^diagnosis ; 


char 


*person_l ; 


char 


*person_2 ; 




} ; 



extern struct student_type student; 
extern char * program_root_dir ; 



void. wtext_out ( HWND wh , int col, int row, char *str, int color); 

/* FUNCTION open_and_pr int_main_menu %/ 

/* REMARKS opens and prints the main window of the student */ 

/* program t , 

/* INPUT HWND *wh - pointer to the window handle */ 

/* int num_ts - number of menu items */ 

/* OUTPUT t/ 

/* CALLED BY main t . 

/* CALLS wtext_out t , 

/%%%%%%%% tt %%%%%%%%%%%% t %t / 

void open_and_pr int_main_menu ( HWND *wh, int num ts) 

{ 

int c ; 



O 
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»wh = WinCreateWindowf 7 , 15, 50, 12, win color , DBL_LIMB_ALL_SIDES , bdr color , EXPLODE!; 
WinCenterText ( *wh , 0, "Sound to Speech", win_color); 
for ( c=0 ; c<num_ts ; c++ ) 

wtext_out ( *wh , text [c 1 . col , text[cl.row, text[cl.str, 



win_color ) 1 



} 



/*******************************************************************/ 

/* FUNCTION assign_central_Eenu_attr ibs »/ 



/* 


REMARKS 


assigns the menu attributes for the main menu to 


*/ 


/* 




allow for point and shoot routine 


*/ 


/* 


INPUT 




*/ 


/* 


OUTPUT 




*/ 


/* 


CALLED BY 


main 


*/ 


/* 


CALLS 


allocate_text_ob ject 


*/ 



/t*****ttttttt**t*t*******x**********t*tttttt***************t**ttttt/ 



void assign_central_menu_attribs ( void ) 

{ 

int lcol = 2 ; 
int first_row=2; 



allocate 

allocate 

allocate 

allocate_ 

allocate 



text_object ( text , 
t ext _obj ect ( text , 
text_object ( text , 
text_obj ect ( text , 
text_object ( text , 



0, STS, lcol, first_row); 

1, SETUP, lcol, f i rst_row+ 2 ) ; 

2, SPEECH_MGR , lcol, f irst_row+4 ) ; 

3, REPORT, lcol, f i rst_row+6 ) ; 

4, EXIT, lcol, f i rs t_row+8 ) ; 



} 






/* 


FUNCTION 


speech__mgr 


*/ 


/* 


REMARKS 


prompts the user to choose between Recognition 


*/ 


/* 




functions and Recording Functions, 


*/ 


/* 


INPUT 


i . 


%/ 


/* 


OUTPUT 




%/ 


/* 


CALLED BY 


main_menu 


%/ 


/* 


CALLS 


allocat e_text_ob j ect 


*/ 






void speech_mgr ( void ) 

{ 

int k = 0 ; 

struct text_type *text; 

HWND wh; 
int c=0; 

#define RECOGNITION "Recognition Functions" 
#define RECORDING "Recording Functions" 




text = (struct text_type*) malloc ( sizeof ( struct text_type) * 3); 
allocate_text_obj ect ( text , 0, RECOGNITION, 2, 2); 
allocate_text_object ( text , 1, RECORDING, 2, 4); 

allocate_text_object ( text , 2, EXIT, 2, 7); 

129 
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wh : WinCreateWinaow( 10 , 27 , 26 , 9 , win_coior, DBL_LI!)E_ALL_3IDES, bdr color, EXPLODE); 
WinCenterText ( wh , 0, "Speech Manager", win_color); 
for (c=0;c<=2;c++) 

wtext_out ( wh , text[c].col, textfcl.row, text [ c 1 . str * 



win _color ) ; 



k = point_and_shoot ( wh , text, 3); 

WinDestroyWindow ( wh ) ; 

for ( c=0 ; c<=2 ; c++ ) 
free ( text [c] .str) ; 
f ree ( text ) ; 
switch ( k ) 

{ 

case 1: status ( spawnlp ( P_WAIT , "strain", 0)); 
break ; 

case 2: status ( spawnlp ( P_WAIT , "speech", 0)); 
break ; 



} /* switch*/ 



} 

status (int val ) 

{ 



if (val = = - 1 ) 

cpr intf ( " f ai led to start child process\n" ) ; 
else 

if (val > 0) cprintf ( "child terminated abnormally\n" ) ; 
delay ( 5000 ) ; 



/************************************** *****************************/ 



/* 


FUNCTION 


main^menu 




*/ 


/* 


REMARKS 


main menu for student program. From here you 


can 


*/ 


/* 




run STSMAIN , SETUP, .SPEECH MANAGER, or return 


to the 


*/ 


/* 




main system menu i 




*/ 


/* 


INPUT 






*/ 


/* 


OUTPUT 






% / 


/* 


CALLED BY 


main 




% / 


/* 


CALLS 


ass ign_central_menu_attribs , open_and_pr int main menu 


*/ 


/* 




speech_mgr 




*/ 



/t*tttttt*tttttt***t*********ttttttt*t**tttt*ttt****t**tttt**t**tttt/ 



void main_menu( void) 

{ 

char buf [80] ; 
int buf len = 80 ; 

HWND wh; 

int item=0, num_ts = 5; 
extern short reverse_color ; 




text 



(struct text_type* ) malloc ( sizeof ( struct text_type) * 5); 



assign_central_menu_attribs ( ) ; 
while (item < num_ts && item != -1 ) 




11/07/1991 



VSWIN.C 



Page 4 



textattr ( reverse_color ) ; 
clrscr ( ) ; 



gotoxy ( 1,1); 

cprintf ( "%s ", getcwd(buf, buflen)); 



open_and_print_main_menu ( &wh , num_ts ) ; 
item = point_and_shoot ( wh , text, num_ts); 
WinDestroyWindow ( wh ) ; 

switch ( item ) 

{ 



case 


-1: item = num_ts ; 








break ; 






case 


1 : spawnlp ( P_WAIT , 

break ; 


"stsmain" 


, 0) 


case 


2 : spawnlp ( P_WAIT , 

break ; 


"setup" , 


0) ; 


case 


3 : speech_mgr ( ) ; 








break; 






case 


4 : spawnlp ( P_WAIT , 


"report" , 


0) ; 



break ; 
} /* switch*/ 



} /*while*/ 



chdir ( program_root_dir ) ; 
f or ( item = 0 ; i tem< = 4 ; i tem++ ) 
free(text[item] .str) ; 
free ( text ) ; 

} 



/ttttttttttttttttttttttttttttttt ** tttttttttttttttttttttttttttttttttt/ 



/* FUNCTION ti tle_screen ( ) 1 */ 
/* REMARKS explodes the title screen */ 
/* INPUT */ 
/* OUTPUT */ 
/* CALLED BY main */ 
/* CALLS get_key() */ 



/*******************************************************************/ 

void title_screen ( void ) 

{ 

HWND wh; 

char ver[5]="", buf [ 1 4 ]= "Version "; 

FILE *fp; 



wh : WinCreateWindow(5, 10, 60, 15, win color , DBL_LINE_ALL_SIDSS , bdr_color, EXPLODE); 
WinCenterText ( wh , 1, "Sound To Speech", win_color); 

fp = f open ( "version . dat" , "rt"); 

if (fp) 

fgets ( ver , 6 , f p ) ; 
fclose ( fp) ; 
strcat(buf, ver); 
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/ttttittttunmmmmmmmmmmmtmmmmmmmmtm/ 

/* WIN - Routines which provide windowing functionality */ 

/* */ 

/* % / 

/* */ 

/tttttitttittttitiititiitttmmmmtmmtimmmmmmimmtt!/ 

/* Modification Log */ 

/* Version Date Programmer Description 

-- */ 

/* */ 

/* V01.00 112787 Bob Withers Program intially complete. */ 

/* */ 

/* */ 



‘include <stdlib.h> 

^include <stddef.h> 

"include <string.h> 

#include "win.h" 

#def ine MAX_WINDOWS 20 



struct 

{ 



sWinData 



BYTE 


cRow ; 


BYTE 


cCol; 


BYTE 


cWidth ; 


BYTE 


cHeight ; 


BYTE 


cWinWidth ; 


BYTE 


cWinHeight ; 


BYTE 


cWinClr ; 


BYTE 


cBdrType ; 


BYTE 


cBdrClr ; 


BYTE 


cCurRow ; 


BYTE 


cCurCol ; 


char 


*pHidden ; 


char 


cSaveData [ 1 



} ; 

typedef struct sWinData WINDATA; 
typedef struct sWinData *PWINDATA; 



static PWINDATA WinHandle [MAX_WINDOWS + 1]; 



/tmmmmmmmttttmmtmmtmttmmtmnmttmmmutt/ 



/* 

/* 

/% 

/* 

/* 

/* 



WinCvt Handle 

Parms : 
hWnd 

Return Value: 



- Convert a window handle into a pointer to the 
window information data structure. 

handle to the window 

pointer to WINDATA or NULL if invalid handle 



/tmmitmmmmtmtttmtmtmtmtttmmmtmtmmttmmt/ 



a 

static PWINDATA near pascal 



WinCvt Handle ( hWnd ) 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 
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HWND 

{ 



hWnd ; 



if ( hWnd < 0 !i hWnd > MAX_WINDOWS ) 
return ( NULL ) ; 
return ( WinHandle [ hWnd] ) ; 



/uiii!!!i!ititiititittit!iiiiiiitiii!i!iiitmmmiimnimmmmm| 

/* WinGetHandle - Return an unused window handle. 

/* 

/* Parms: None 

/* 

/ * Return Value: Window handle or NULL if none available 

/tmmmmmmmmsmtsmmmtmtmmmmtmimmmmt/ 



static HWND near pascal WinGetHandle ( ) 

{ 




} 



register int i; 

for (i = 1; i <= MAX_WINDOWS; ++i) 

{ 

if (NULL == WinHandle [il ) 
return ( i ) ; 

} 

return ( NULL ) ; 



/ittmmttttttiittiittmiiiitiimtiittimitmmitmmtmitmtim/ 



/* 


WinExplodeWindow - Draws an exploded window on the screen. 


/* 






/* 


Parms : 




/* 


nRow - ' 


Top row of requested window (1 relative) 


/* 


nCol - ' 


Left column of requested window (1 relative) 


/* 


nWidth - Width (in columns) of requested window 


/* 


nHeight 


- Height (in rows) of requested window 


/* 


nWinClr 


- Color of the window background 


/* 


nBdrType 


- Type of border for this window (defined 


*/ 






/* 




N0_B0X 


/* 




DBL_LINE_TOP BOTTOM 


/* 




DBL_LINE_SIDES 


/* 




DBL_LINE_ALL_SIDES 


/* 




SNGL LINE ALL SIDES 


/* 




GRAPHIC_BOX 


/* 




N0_WIND_B0RDER 


/* 


nBdrClr 


- Color or the window border 


/* 






/* 


Return Value: 


None 



/iitititiitiiitttttttittiiiimmmmimmmmmmmmmmmti/ 



void near 



ERIC 



short 

short 

{ 



pascal WinExplodeWindow ( nRow , nCol , nWidth, nHeight, 

nWinClr, nBdrType, nBdrClr) 
nRow, nCol , nWidth, nHeight; 
nWinClr , nBdrType, nBdrClr; 




*/ 

*/ 

*/ 

*/ 

*/ 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

WIN.H) 

*/ 

*/ 

*/ 

’ */ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 
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} 



register short nLRR , nLRC , nXl , nYl, nX2 , nY2 ; 

nLRR = nRow + nHeight - 1 ; 

nLRC = nCol + nWidth - 1 ; 

nXl = (nRow + (nHeight >> 1)) - 1; 

nYl = (nCol + (nWidth >> 1)) - 3; 

nX2 = nXl + 2; 

nY2 = nYl + 5; 

while (TRUE) 

{ 

ScrClearRect ( nX 1 , nYl, nY2 - nYl + 1, nX2 - nXl + 1, nWinClr); 
ScrDrawRect ( nXl , nYl, nY2 - nYl + 1, nX2 - nXl +1, 
nBdrClr, nBdrType ) ; 

if ( nXl = = nRow && nYl == nCol && nX2 == nLRR && nY2 == nLRC) 
break ; 

nXl = (nXl - 1 < nRow) ? nRow : nXl - 1 ; 

nYl = (nYl - 3 < nCol) ? nCol : nYl - 3; 

nX2 = ( nX2 + 1 > nLRR) ? nLRR : nX2 + 1; 

nY2 = ( nY2 + 3 > nLRC) ? nLRC : nY2 + 3; 

} 

return ; 



/ttiiiiitttttitittitiitititimiiimimmmmimtmmmmmmm/ 

/* tfinDrawKindow - Draws a window on the screen without creating the »/ 



/* 




WINDATA structure or saving the previous screen 


/* 




contents . 


/* 






/* 


Parms : 




/* 


nRow 


Top row of requested window (1 relative) 


/% 


nCol 


Left column of requested window (1 relative) 


/* 


nWidth 


Width (in columns) of requested window 


/% 


nHeight 


- Height (in rows) of requested window 


/* 


nWinClr 


- Color of the window background 


/* 


nBdrType 


- Type of border for this window (defined 


*/ 




/* 




NO_BOX 


/* 




DBL_LINE_TOP_BOTTOM 


/* 




DBL_LINE_S IDES 


/* 




DBL_LINE_ALL SIDES 


/* 




SNGL LINE ALL SIDES 


/* 




GRAPHIC_BOX 


/* 




NO_WIND_BORDER 


/* 


nBdrClr 


- Color or the window border 


/* 


bExplodeWin 


- Boolean value requesting either a pop-up or 


/* 




exploding window 


/* 




TRUE ==> Exploding window 


/* 




FALSE ==> Pop-up window 


/* 




/* 


Return Value: 


None 



/mmmmtmmummmtmmmtmmmmmmmmmmmt/ 




in 



*/ 

*/ 

*/ 

%/ 

*/ 

*/ 

*/ 

*/ 

*/ 

WIN . H ) 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



void pascal WinDrawWindow ( nRow , nCol, nWidth, nHeight, 
q nWinClr , nBdrType, nBdrClr, bExplodeWin ) 

short nRow, nCol, nWidth, nHeight; 

134 
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i 



short 

short 

{ 



nWinClr, nBdrType , nBdrClr; 
bExplodeWin ; 



if (bExplodeWin) 

WinExplodeWindow{ nRow , nCol , nWidth 

nWinClr, nBdrType, 



else 



nHeight 
nBdrClr ) 



1 



{ 

ScrClearRect ( nRow , nCol , nWidth, nHeight, nWinClr); 
ScrDrawRect ( nRow , nCol , nWidth, nHeight, nBdrClr, nBdrType); 

} 

return ; 



/tmtitmmimmmimtimmmmmmmiiimmttmitmumt; 
/* WinCr eateWindow - Create a window with the requested attributes and */ 



/* 




return a handle which may be used to identify this 


*/ 


/* 




particular window in future calls. 


*/ 


/* 


Parms : 




*/ 


/* 


nRow 


Top row of requested window (1 relative) 


*/ 


/* 


nCol 


Left column of requested window (1 relative) 


*/ 


/* 


nWidth 


Width (in columns) of requested window 


*/ 


/* 


nHeight 


- Height (in rows) of requested window 


*/ 


/% 


nWinClr 


- Color of the window background 


*/ 


/* 
± / 


nBdrType 


- Type of border for this window (defined in 


WIN . H ) 


/* 




NO_BOX 


*/ 


/* 




DBL_LINE_TOP BOTTOM 


*/ 


/* 




DBL_LINE_SIDES 


*/ 


/* 




DBL_LINE_ALL SIDES 


*/ 


/* 




SNGL LINE ALL SIDES 


*/ 


/* 




GRAPH IC_BOX 


*/ 


/* 




NO_WIND_BORDER 


*/ 


/* 


nBdrClr 


- Color 'or the window border 


*/ 


/* 


bExplodeWin 


- Boolean value requesting either a pop-up or 


*/ 


/% 




exploding window 


*/ 


/% 




TRUE ==> Exploding window 


*/ 


/% 




FALSE ==> Pop-up window 


*/ 


/* 






*/ 


/* 


Return Value : 




% / 


/* 


hWnd 


Handle of the created window or NULL if an error 


X/ 


/* 




prevented the creation 


X/ 



/titiitttttttittttmmmmmmmmtmtmtmmmtmtmmmtt/ 




HWND pascal WinCreateWindow( nRow , nCol , nWidth, nHeight, 

nWinClr , nBdrType, nBdrClr, bExplodeWin) 
short nRow, nCol, nWidth, nHeight; 

short nWinClr, nBdrType, nBdrClr; 

short bExplodeWin; 




register PWINDATA 
auto HWND 



pWinData ; 

hWnd ; 




hWnd = WinGetHandle ( ) ; 
if (NULL == hWnd ) 
return ( hWnd ) ; 
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} 



pWinData = (PWINDATA) 

if ((PWINDATA) NULL ! 

( 

WinHandle [hWndl = 
pWinData->cRow = 

pWinData- > cCol = 

pWinData->cWidth = 
pWinData->cHeight = 
pWinData->cWinClr = 
pWinData->cBdrType = 
pWinData->cBdrClr = 
pWinData- > cCurRow = 
pWinData->pHidden = 
if ( NO_WIND_BORDER ! 
{ 



malloc(sizeof( WIND AT A ) 

+ ScrGetRectSize ( nWidth , nHeight)); 
pWinData ) 

pWinData; 

( BYTE ) nRow ; 

(BYTE) nCol; 

pWinData- > cWinWidth = (BYTE) nWidth; 
pWinData->cWi nHeight = (BYTE) nHeight; 
(BYTE) nWinClr; 

(BYTE) nBdrType ; 

(BYTE) nBdrClr; 

pWinData- > cCurCol = (BYTE) 1; 

NULL; 
nBdrType ) 



pWinData->cWinWidth -= 2; 
pWinData->cWinHeight -= 2; 

} 

ScrSaveRect ( nRow , nCol, nWidth, nHeight, pWinData->cSaveData ) 
if ( bExplodeWin ) 

WinExplodeWindow ( nRow , nCol , nWidth, nHeight, 

nWinClr, nBdrType, nBdrClr); 

else 



{ 

ScrClearRect ( nRow , nCol , nWidth 
ScrDrawRect (nRow, nCol , nWidth, 

} 

WinSetCursorPos ( ( HWND ) pWinData, 1, 

} 

return ( hWnd ) ; 



nHeight, nWinClr); 
nHeight, nBdrClr, nBdrType); 

1 ) ; 



It WinDestoryWindow - Destroy the window represented by hWnd and replace */ 

/* the previous screen contents saved when the window */ 



/* was created. %/ 

/* Parms : t / 

/* hWnd - Handle to the window to be destroyed * / 

/* */ 
It Return Value: TRUE :> window destroyed, FALSB :> invalid handle */ 

Itttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttutttttuttttti 



BOOL pascal WinDestroyWindow ( hWnd ) 

HWND hWnd; . 

{ 

register PWINDATA pWinData; 




pWinData = WinCvtHandle ( hWnd ) ; 
if (NULL == pWinData) 
return) FALSE) ; 

ScrRestoreRect ( pWinData->cRow, pWinData->cCol , pWinData- >cWidth , 
pWinData- > cHeight , pWinData->cSaveData ) ; 
if (NULL != pWinData- > pHidden ) 
free ( pWinData- > pHidden ) ; 

138 
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} 



free ((char *) pWinData); 
Winllandle [ hWnd ] = NULL; 
return ( TRUE ) ; 



/mtmmmmmtmmmmmmmmtmmmmmmmmmm/ 

/ * WinScrollWindowUp - Scrolls the requested window up one line. 
/* 

/* Parms : 

/* hWnd - Handle to the window to be scrolled 

/* 

/* Return Value: None 

/mmmmmmmmummmmmmtmmummmmmmmt/ 



void pascal WinScrollWindowUp ( hWnd ) 

HWND hWnd ; 

{ 

register PWINDATA pWinData; 

auto short nRow, nCol; 




} 



pWinData = WinCvtHandle ( hWnd ) ; 
if (NULL == pWinData) 
return ; 

if (NULL == pWinData->pHidden ) 

{ 

nRow = pWinData- >cRow ; 
nCol = pWinData->cCol ; 

if ( NO_WIND_BORDER != pWinData->cBdrType ) 

{ 

nRow++ ; 
nCol ++ ; 

} 

ScrScrollRectUp ( nRow , nCol , s pWinData->cWinWidth , ' 

pWinData->cWinlIeight , 1, pWinData->cWinClr ) ; 

return ; 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



/tmmmitmmtmtmmmmtitmmtmimtmtmtmttmtmit/ 

/* WinScrollWindowDown - Scrolls the requested window down one line 
/* 

/* Parms: 

/* hWnd - Handle to the window' to be scrolled 

/* 

/* Return Value: None 

/mmmmmmmtmmtmmmmtttmmmtmttmtmmmm/ 



void 

HWND 



pascal WinScrollWindowDown ( hWnd ) 
hWnd ; 



register PWINDATA pWinData; 

auto short nRow, nCol; 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 




pWinData = WinCvtHandle ( hWnd ) ; 
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} 



if (NULL == pWinData) 
return ; 

if (NULL == pWinData- >pHidden ) 

{ 

nRow = pWinData->cRow; 
nCol = pWinData->cCol ; 

if ( N0_WIND_B0RDER != pWinData->cBdrType ) 

{ 

nRow++ ; 
nCol++ ; 

} 

ScrScrollRectDown ( nRow , nCol , pWinData->cWinWidth , 

^ pWinData->cWinHeight , 1, pWinData->cWinClr ) ; 

return ; 



/tttttttttttttttmttmmittmiimiimmmitimtmmtmmmtm/ 



/t WinSetCursorPos 

/* 

/* 

/* Parms : 


- Position the 


cursor relative to the selected window. */ 

The upper left hand corner of the 


/* 


hWnd 


- 


Handle to the window to position 


/* 


nRow 


- 


Row to position cursor to within 


/* 

/* 


nCol 




Col to position cursor to within 


/* 


Return 


Value : 


None 



/mmimmmmmmmtmmimimmtmmmmmmtmmm/ 




window is 



the cusor 
window ( 1 
window ( 1 



( 1 , 1 ) 



in 

relative ) 
relative ) 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



void pascal WinSetCursorPos ( hWnd , nRow, nCol) 

HWND hWnd ; 

short nRow, nCol; 

{ j 

register PWINDATA ‘pWinData; 

auto short nMaxRow, nMaxCol; 




if (NULL == hWnd ) 

{ 

ScrSetCursorPos ( nRow , nCol); 
return ; 

} 

pWinData = WinCvtHandle ( hWnd ) ; 
if (NULL == pWinData) 
return ; 

if (nRow > pWinData- >cWinHeight && nCol > pWinData- >cWinWidth ) 
return ; 

pWinData->cCurRow = (BYTE) nRow; 

pWinData->cCurCol = (BYTE) nCol; 

nRow = nRow + pWinData- >cRow - 1; 

nCol = nCol + pWinData- >cCol - 1; 

if ( N0_WIND_B0RDER != pWinData- > cBdrType ) 

{ 

++nRow ; 

++nCol ; 

} 
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ScrSetCursorPos ( nRow , nCol); 
return ; 



/ttmtmtmttttttmttmtttmuttttttttttttmmtmtmtttttimtutt/ 

/ * WinClearScreen - Clear a window to the desired color. %/ 

/* */ 

/ * Parms : * / 

/* hWnd - Handle to the window to be cleared < */ 

/* (A handle of NULL clears the entire screen) %/ 

nColor - Color to be used in clearing the window %/ 

/* %/ 

/* Return Value: None %/ 

/tiituiiiiiiiimummimmtimtmmtmitmmimumimmm/ 



void pascal WinClearScreen ( hWnd , nColor) 

HWND hWnd; 

short nColor ; 

{ 

register PWINDATA pWinData; 

auto short nRow, nCol ; 

if (NULL == hWnd ) 

ScrClearRect ( 1 , 1, 80, 25, nColor); 




pWinData = WinCvtHandle ( hWnd ) ; 
if (NULL == pWinData) 
return ; 

nRow = pWinData->cRow; 

nCol = pWinData->cCol ; 

if ( NO_WIND_BORDER != pWinData->cBdrType ) 

{ f 

++nRow; !' 

++nCol ; 

} 

pWinData->cWinClr = (BYTE) nColor; 

ScrClearRect ( nRow, nCol , pWinData- >cWinWidth , pWinData->cWinHeight 
pWinData->cWinClr ) ; ’ 

} 

return ; 

) 



/ittitttttittitiitittittttittttitiitmmmmmimtimimmmmiii/ 

/* WinTextOut - Display a string to the requested window at 

/ 



/* 

/* 

/* 

/* 

/* 

/* 

/* 

/% 

ERIC 7 




Parms : 
hWnd 
pStr 
nAttr 



current cursor location (for that window) using the 
passed color attribute. 

If the string extends beyond the boundries of the 
window it will be truncated. 



Handle of the window 

Pointer to the NULL terminated 

Color attribute to.be used in 



string to 
displaying 



display 
the string 



the 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 
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/ 



/* Return Value: None 

imtmmiimmmmmmmmmttmtmmtt 



mmmmmmtt 



*/ 



void pascal WinTextOut ( hWnd , pStr, nAttr) 

HWND hWnd ; 

char *pStr; 

short nAttr; 

{ 

register PWINDATA pWinData; 

auto short nCount; 

auto short nRow, nCol ; 

pWinData = WinCvtHandle ( hWnd ) ; 
if (NULL == pWinData) 
return ; 

ScrGetCursorPos ( &nRow , &nCol); 

WinSetCursorPos ( hWnd , pWinData->cCurRow , pWinData- > cCurCol ) ; 
nCount = pWinData-icWinWidth - pWinData-icCurCol + 1; 
ScrTextOut ( pStr , nAttr, nCount); 

ScrSetCursorPos ( nRow, nCol); 
return ; 

} 



*/ 



/tittttttttititttttittiitmmttmitmimiimmmmmtmittmtm/ 



/* 


WinCenterText 


- Centers a text string in a window. 


*/ 


/* 






*/ 


/* 


Parms : 




*/ 


/* 


hWnd 


Handle of the window 


*/ 


/* 


nRow 


Window row to place the string on 


*/ 


/* 


pStr 


Pointer to the string to be displayed 


*/ 


/* 


nColor 


Color attribute used to display the string 


*/ 


/* 






*/ 


/* 


Return Value: 


None 

? ; 


*/ 



/mmmmmmmmmmmmmmmmmmmmmmmmm/ 




void pascal WinCenterText ( hWnd , nRow, pStr, nColor) 

HWND hWnd; 

short nRow; 

char *pStr; 

short nColor; 

{ 

if (NULL == WinCvtHandle ( hWnd ) ) 
return ; 

WinSetCursorPo s ( h Wnd , nRow, ( WinGetWindowWidth (hWnd ) - strlen(pStr) ) / 2); 
WinTextOut ( hWnd , pStr, nColor); 
return ; 

} 




O 



/mmmmmmmmmmmmmmmmmmmmmmmmm/ 

/* WinMoveWindow - Move an existing window to a new screen location. */ 
/* In this version the window to be moved MUST be fully */ 

/* visible on the screen for WinMoveWindow to perform */ 

/* properly. If the window being moved is completely or * / 

/* partially under another window the screen will not */ 

/* be left in the correct state (i.e. garbage on screen).*/ 
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/* 




/* 




/* 




/* 




/* 




/* 




/* 


Parms : 


/* 


hWnd 


/* 


nRow 


/* 


nCol 


/* 




/* 


Return 



Value : 



It is the callers responsibility to insure that the 
window is not being moved off the screen. Even with 
these restriction this can be a handy routine and is 
included for that reason. A future release of the 
package may fix these shortcomings. 



Handle to the window to be moved 
Move the window to this row 
Move the window to this column 



None 







*/ 
*/ 
*/ 
*/ 
% / 
*/ 
*/ 
*/ 
*/ 
*/ 
*/ 
*/ 



void pascal WinMoveWindow ( hWnd , nRow, nCol) 
HWND hWnd; 

short nRow, nCol ; 

{ 

register PW INDATA pWinData; 

register char *pBuf; 




} 



pWinData = WinCvtHandle ( hWnd ) ; 
if (NULL == pWinData) 
return ; 

if (NULL != pWinData->pHidden ) 

{ 

pWinData->cRow = (BYTE) nRow; 
pWinData- >cCol = (BYTE) nCol ; 
return ; 

} 

— ma 1 1 oc ( S c rGe tRec t S i z e ( pW i nDa t a — > c W i d t h , pWinData— ^cHeisht ) ) ■ 
if (NULL ! = pBuf ) ’ 

{ 

ScrSaveRect ( pWinData->cRow, ■„ pWinData- > cCol , 

pWinData->cWidth , pWinData->cHeight , pBuf); 

ScrRestoreRect ( pWinData- >cRow , pWinData->cCol , 

pWinData->cWidth , pWinData->cHeight , 
pWinData->cSaveData ) ; 

ScrSaveRect ( nRow , nCol , pWinData->cWidth , pWinData->clIeight , 
pWinData- > cSaveData ) ; 

ScrRestoreRect ( nRow , nCol, pWinData- > cWidth , pWinData->cHeight nBufi- 
pWinData- > cRow = (BYTE) nRow; w v ’ 

pWinData- >cCol = (BYTE) nCol ; 
f ree ( pBuf ) ; 

} 

return; 



/ttmmmmmmmmmtmmmmmmutmmmmmmmm/ 

/* WinGetWindowEow - Returns the row value currently associated with the */ 

passed window handle. 



/* 

/* 

/* 

/% 

/* 



ERIC 



Parms : 
hWnd 



- Handle to the window 



*/ 

*/ 

*/ 

*/ 

*/ 



Return Value: 



Row the window currently resides at 
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/ 






tt 



tti 



short pascal WinGetWindowRow ( hWnd ) 

HWND hWnd ; 

{ 

register PWINDATA pWinData; 



pWinData = WinCvtHandle ( hWnd ) ; 
if (NULL == pWinData) 
return ( 0 ) ; 

return ( pWinData->cRow) ; 



/ittiiitmiitittmtiitttimitiiitiimiitmmimtmmiimmmitit/ 

/* WinGetWindowCol - Returns the col value currently associated with the t/ 

/ * passed window handle. 

/* Parms: 

/ * hWnd - Handle to the window 

/* 

/ * Return Value: Column the window currently resides at 

/mmmmmttmmmmmmtmtmmmmmmmmmmitm/ 



short pascal WinGetWindowCol ( hWnd ) 
HWND hWnd ; 




{ 



register PWINDATA 



pWinData ; 



*/ 

*/ 

*/ 

*/ 

*/ 



pWinData = W’inCvtHandle ( hWnd ) ; 
if (NULL == pWinData) 
return ( 0 ) ; 

return ( pWinData->cCol ) ; 



i 



i . 



/mmmmmtmmmmmmmmmmmmmmmmnmmm/ 

/* WinGetWindowWidth - Returns the column width of the passed window 
/* 

/* Parms: 

/* . hWnd - Handle to the window 

/* 

/* Return Value: Number of columns in the window 

/mmmmmmmmmmmmmmtmmmmmmmmtmim/ 

short pascal WinGetWindowWidth ( hWnd ) 

HWND hWnd 

{ 

register PWINDATA pWinData; 




pWinData = WinCvtHandle ( hWnd ) ; 
if (NULL == pWinData) 
return ( 0 ) ; 

return ( pWinData->cWinWidth ) ; 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



O 

ERLC 
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/tittttitimtttitittmittiitttmtitttmmtimtmmmmtmtmmi/ 

/* WinGetWindowHeight - Returns the nuaber of rows in the passed window, t/ 

/* */ 

/ * Parms : * / 

/ * hWnd - Handle to the window */ 

/ * % / 

/* Return Value: Number of rows in the window */ 

/mttmmmmmmmmmmmmmttmmmtmtmmmmm/ 



short pascal WinGetWindowlIeight ( hWnd ) 

HWND hWnd; 

{ 

register PWINDATA pWinData; 

pWinData = WinCvtHandle ( hWnd ) ; 
if (NULL == pWinData) 
return ( 0 ) ; 

return ( pWinData- >cWinIIeight ) ; 

} 



/ttttimmmmmttmmmimtmtmuttmtttmmmmmittmi/ 

/ * W'inGetWindowClr - Get the window background color 
/* 

/* Parms: 

/* hWnd - Handle to the window 

/* 

/* Return Value: Returns the attribute for 

/mmmmmmmmmmmtmmmmmmmmummmmm/ 



the window color 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



short pascal WinGetWindowClr ( hWnd ) 

HWND hWnd; 

{ 

register PWINDATA .pWinData; 

^ : 

pWinData = WinCvtHandle ( hWnd ) ; 
if (NULL == pWinData) 
return ( 0 ) ; 

return ( pWinData->cWinClr ) ; 

} _ 



/niitttntttttitititttmttmmimmmmmmmmmmmitmtit/ 

/* WinGetWindowBdrClr - Get the window border color 
/* 

/* Parms: 

/* hWnd - Handle to the window 

/* 

/* Return Value: Returns the attribute for the window border color 

/tmmmmtttmtmtmttttttutmttmtttttttmtmmmmtmmt/ 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



short 

HWND 

{ 



pascal WinGetWindowBdrClr ( hWnd ) 
hWnd ; 




register PWINDATA 



pWinData ; 
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} 



pWinData = WinCvtHandle ( hWnd ) ; 
if (NULL == pWinData) 
return ( 0 ) ; 

return ( pWinData- >cBdrClr ) ; 



/mmmmmmmmmmmmmmmmmmmmmmmmm/ 

/* WinGetBorderType - Gets the border type of the passed window 
/* 

/* Parras: 

/* hWnd - Handle to the window 

/* 

/* Return Value: Returns the window border type 

/mtmmmmmmmmmmmmmmmmmmmmmmmu/ 

short pascal WinGetBorderType ( hWnd ) " — 

HWND hWnd; 

{ 

register PWINDATA pWinData; 

pWinData = WinCvtHandle ( hWnd ) ; 
if (NULL == pWinData) 
return ( 0 ) ; 

return ( pWinData- >cBdrType ) ; 




/mmmmmmmmmmmmmmmmmmmmmmmmm/ 

/* WinHiaeWindow - Removes a window from the screen, saving it’s */ 



/* 

/* 

/* 

/* 

/* 

/* 

/% 

/* 



Parms : 
hWnd 

Return Value 



contents . The window can later be placed back on 
the screen via WinShowWindow( ) . Note that in this 
release the window MUST be fully visible for this 
operating to work correctly. 

i r 

- Handle to the window 

TRUE => window hidden, FALSE => buf alloc failed 



/mmmmmmmmmmmmmmmmmmmmmmmmm/ 



BOOL pascal WinllideWindow ( hWnd ) 
HWND hWnd; 

{ 

register PWINDATA 
auto char 

auto short 

auto short 



pWinData ; 

*pBuf ; 
nBufSi ze ; 
nRow , nCol , 



nWidth, nHeight 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 



*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 

*/ 




pWinData = WinCvtHandle ( hWnd ) ; 
if (NULL == pWinData) 
return ( FALSE ) ; 
nRow = pWinData- > cRow ; 

nCol = pWinData- > cCol ; 

nWidth = pWinData->cWidth ; 
nHeight = pWinData->cHeight ; 
nBufSize = ScrGetRectSize ( nWidth , nHeight); 
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} 



if (NULL != pWinData->pHidden ) 
f ree ( pWinData->pHidden ) ; 
pBuf = malloc ( nBuf Size ) ; 

if (NULL == pBuf ) 
return ( FALSE ) ; 

ScrSaveRect ( nRow , nCol, nWidth, nlleight, pBuf ) ; 

ScrRestoreRect ( nRow , nCol, nWidth, nlleight, pWinData->cSaveData ) ; 
pWinData->pHidden = pBuf ; 
return ( TRUE ) ; 



/ummtmiimmmmmmmimmmmumiimmmiimmmi 

/* WinShowWindow - Places a hidden window back on the screen and frees »/ 

/* the buffer used to hold the window image. 

/* Parms : 

/ * hWnd - Handle to the window 

/* 

/* Return Value: TRUE :> window shown, FALSE window wasn’t hidden */ 

/mnmmmmmmtmtmtmmmmmmmmmtmmmmm/ 



BOOL pascal WinShowWindow ( hWnd ) 

HWND hWnd ; 

{ 

register PWINDATA pWinData; 




} 



pWinData = WinCvtHandle ( hWnd ) ; 
if (NULL == pWinData) 
return ( FALSE ) ; 

if (NULL == pWinData->pHidden ) 
return ( FALSE ) ; 

ScrRestoreRect ( pWinData- > cRow , pWinData->cCol , pWinData- >c Width , 
pWinData->cHeight , pWinData- > pHidden ) ; 
f ree ( pWinData- >pHidden ) ; , ; 

pWinData->pHidden = NULL; }• • 

return ( TRUE ) ; 



*/ 

*/ 

*/ 

*/ 



/itttitttmutttittuittttttttutttttmtmmmtmmtmmmtimm/ 

/* Winlnitialize - Init the windowing system. 
/* 

/* Parms: None 

/* 

/* Return Value: None 

/mttmmmumtttmmmmmtmtutmmtttmtmimmtttmt/ 



void pascal Winlnitialize ( ) 



{ 




} 



Scrlnitialize ( ) ; 

memset ( ( char *) WinHandle, 

return ; 



NULL, 



sizeof ( WinHandle ) ) ; 



*/ 

*/ 

*/ 

*/ 

*/ 
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/* WinTerminate - Clean up the windowing package 
/* 

/* Parms : None 

/* 

/* Return Value: None 

/mmmmmmmmmmmmmmmmtmmmmtmmmmt/ 

void pascal WinTerminate ( ) 

{ 

register short i; 

for (i = 1; i <= MAX_WINDOWS; ++i) 

{ 

if ( WinHandle [ i ] != NULL) 

WinDestroyWindow ( i ) ; 

} 

return; 

) 



*/ 

*/ 

*/ 

*/ 

*/ 



s 
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The ARC’S Bioengineering Program is currently involved in research utilizing voice 
recognition systems designed to allow individuals with mental retardation and/or severe 
handicaps to control their environment and communicate with others. The most recent 
development in this research track is the Sound-to-Speech Translations Using Graphics 
Symbols (STS/Graphics) computer-based system which incorporates a photographic- 
quality graphics display. The STS/Graphics system accepts keyboard input and 
delivers environmental control and digitized speech output. Since 1987, the system has 
undergone three rounds of alpha testing and one round of beta testing resulting in 
system refinements which increase speed of output and ease of training the system to 
recognize the user’s vocalizations. 

The system was developed to enable persons with mental retardation to use their own 
voices to communicate. Utilizing the photographic quality images with the system 
turns a simple communication board into a communication system which the user can 
operate using their own speech. 

Funding for the Sound-To-Speech system development was provided through Grant 
No. H180P90015 from the U.S. Department of Education. 
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THE SOUND-TO-SPEECH SYSTEM 



What Is It and Who Is It For? 

The Sound -To-Speech system is an electronic system which enables persons who have 
unintelligible speech to more effectively communicate and control their environment. 
Individuals may be nonspeaking due to a variety of impairments which make their 
speech difficult for all but familiar listeners or communication partners to understand. 
Additionally, these persons may experience concomitant physical and/or cognitive 
differences which further compromise their independence. 

Today, there are a variety of commercially available communication aids for persons 
who may benefit from the use of an augmentative communication system. 

Traditionally these systems require operation using a switch or by pointing to the 
display surface. There is an absence of systems which utilize the user's own voice for 
communication. Voice-recognition systems which are currently available require the 
user to control them via intelligible speech commands. The product of these commands 
is typically text production. These systems are mostly widely used by industry or the 
military. Text is printed on the computer screen or other multi-task steps. These 
systems present barriers for persons who are unable to read, or are physically and/or 
cognitively impaired. 

The Sound-To-Speech System was designed to enable nonspeaking persons with a 
variety of abilities to use their own voice to communicate and/or control their 
environment. Furthermore, the sound-to-speech system is not a test-based system. 
Communication displays are available which can be customized to meet the user’s 
needs. Photographs are taken of desired items from the user’s environment and 
scanned into the system. These scanned images can then be placed in the user’s 
communication displays and appear as photographic quality images. The system also 
assists persons with physical impairment to control their environment by enabling them 
to control simple household appliances (i.e. TV, VCR, radio, lights etc.). 
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The system is unique for the following reasons: 

1. Enables the user to use their own voice to communicate 

2. The system translates unintelligible speech sounds in to meaningful speech 
messages which are spoken by the computer 

3. Use of photographic quality images on the communication displays which 
represent choices/activities in the user’s natural environment 

4. Speech output that is digitized easily understood, age and gender appropriate 

5. Data on system use is automatically recorded, thereby facilitating the 
development of appropriate training strategies 
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System Control 

Users can control the system in two ways: scanning and direct selection. 

Scanning Using their voice as a switch, the user may stop a highlighted box when it 
reaches a desired item on the display. Two scanning motions are available in the 
STS/Graphics software which include linear and row/column scanning. 

Selection of either mode is based on the user’s cognitive and physical abilities. In each 
case, the system requires a single vocalization. Vocalizations are not required to be 
consistently similar in their production. Once the scanning action is stopped by the 
user, the functions associated with that picture image are executed which may include a 
spoken message and/or environmental control function. 

Some individuals are unable to produce a variety of sounds for each message they wish 
to convey and present physical abilities which prohibit their ability to point to items on 
a communication display. Using their voice as a switch they are able to look at a 
communication display and choose the item from the display as it is presented by the 
computer. They use their own voice as a switch to indicate their choice to the 
computer which then provides their message and/or activates an appliance in their 
environment. 

For individuals who require and prefer the use of a switch, the system has been adapted 
to accept any switch with a 1/8" stereo plug. 

Direct selection Using different and distinct, yet unintelligible vocalizations, an 
individual may select any item appearing on the display. The system is configured to 
recognize each individual user’s "voice". Information is stored for each individual 
which enables the system to recognize their speech when their display is presented on 
the monitor. In this mode, the user chooses a sound or vocalization which will be used 
to retrieve a message from the system. For example, the sound "ma" may mean "I 
want to listen to music." An image of a radio or tape recorder may appear on the 
display in front of them. By producing the sound "ma" the system will respond by 
producing the message "I want to listen to music" and/or activation the radio or tape 
recorder. 
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Display Sizes 

Displays can be created of varying sizes ranging from 2x2toa5x5 matrix. The size 
of the display and the number of images used is dependent on the needs and abilities of 
the user. For each item chosen for the display, a spoken message and/or environmental 
control function can be selected. For example, a photograph of a tape player might be 
configured to produce the message "I like this music" as well as activating the tape 
recorder to play music. 



Customizable User Options 

A variety of options are available on the system which enhance its customization for the 
user. They include: 

Scan Speed The scan speed in the linear or row/column scanning mode can be 
adjusted to the user’s needs. 

Recognition Level the recognition level of the system can be "tuned" to recognize the 
distinct features of the user’s speech sounds. How distinctly these sounds must be 
produced to be recognized by the system may be adjusted to accommodate the user’s 
abilities. 

Matrix and Scan Box Color The color of the display or matrix and the scan box may 
be changed. Field test studies with the STS system determined that for some users, the 
contrast between the matrix and scan box color was important to their ability to attend 
and visually track activity on the display. 

Auditory Signal In the scanning mode, the system has the capability to provide a beep 
or signal as it advances from one image to the next. This signal can be turned on or off 
while the display is in use. For some of the field test participants the signal was needed 
to draw their attention to the display as well as assist in visual scanning and tracking 
activity on the display. 

Speech Output 

The STS system produces digitized speech messages in response to a user’s speech 
sound. Digitized speech sounds very much like a recording of human speech and can 
be customized to the user’s age and gender. It is highly intelligible and is most 
effective with user’s who present cognitive impairment. 

Recordings of the user’s messages can be provided by an individual who is similar in 
age and gender and stored in the system. When the user produces a speech sound, the 
accompanying recorded message is heard by the communication partner. 
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INSERT ILLUSTRATION OF REPORT SUMMARY HERE 



Date/Time: Wed Jan 29 09:14:37 1992 
Name: Maggie Sauer 
Age: - 

Diagnosis: - 
Assistant 1 : - 
Assistant 2: - 
Location: - 



DISPLAY CONFIGURATION 

Access Method: Direct Select £ 

Page: DIRECT 

Matrix Size: 3x3 

Initial Recognition Distance: 50 

Matrix Color: Black 

Scan Color: Green 



The Arc 

SOUND-TO-SPEECH 



STUDENT SUMMARY 



CELL INFORMATION 



Cell 1: 
Synonym: 



Message: 0 



Cell 2: LIPGLOSS 
Synonym: LIPGLOSS 



Message: 3 



Cell 3: 
Synonym: 



Message: 0 



Cell 4: REDPOL 
Synonym: REDPOL 



Message: 1 



Cell 5: 
Synonym: 



Message: 0 



Cell 6: LOTION 
Synonym: LOTION 



Message: 2 



Cell 7: 
Synonym: STIC 



Message: 0 



Cell 8: STICKER 
Synonym: STIC 



Message: 4 



Cell 9: 
Synonym: STIC 



Message: 0 
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Data Collection 

A chronological report is generated of all system activity whenever the system is used. 
Subsidiary reports are generated from the chronological report detailing specific 
activities such as environmental control audio playback, the number of times each 
selection was made and a summary of the display setup. The subsidiary reports include 
an analysis of these activities. 

This method of data collection assists those providing support to the user and the 
system to look more carefully at user performance and provide appropriate training 
strategies. It also allows the user’s performance to be documented over time. 



Multiple User Memory 

The system has the capability to store communication displays for a variety of users. 
Thus, the STS system can be customized for one or more users. However, only one 
person may use the system at one time. The system is not able to recognize the speech 
sounds of more than one user at a time. { 



Appropriate Candidates for the STS System 

Individuals with a variety of presenting etiologies field tested the STS system. Some of 
these included cerebral palsy, autism, mental retardation and Down’s Syndrome. Most 
often, these individuals were described as "very talkative" regardless of the 
intelligibility of their speech. In some cases, a variety of other systems which required 
them to point to a picture/display or use a switch had been tried. In each case, they 
preferred to rely on speech and other communication strategies such as gestures, 
eyegaze, facial expression and interpretation by a familiar communication partner. 

From these experiences, a list of user characteristics was generated: 

1 . adequate vision and hearing 

2. adequate positioning and seating 

3. understanding of picture/object associations 

4. possess rudimentary scanning skills using a switch (if appropriate) 

5. speech is not functional as a primary mode of communication 
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Considerations for Use of the STS System 

While the considerations outlined below are not meant to preclude a potential user from 
operation of the STS system, they are provided to suggest areas which can make the 
difference between optimal and inconsistent performance. 

Seating/Positioning Appropriate seating and positioning in a chair, wheelchair etc. is 
essential to effective system operation. These arrangements will permit the user to best 
utilize his/her body to control a switch (if appropriate) , body posture for phonation 
and speech production and monitoring system operation/communication displays. 

It is advised that if effective seating and positioning arrangements have not been 
obtained, an evaluation by a physical and occupational therapist may assist in these 
determinations. These recommendations can facilitate effective system use and/or 
switch operation. 

Cognitive Considerations The STS system incorporates the use of photographs from 
the user’s natural environment into the system. Additionally, the photographs can be 
displayed in a variety of sizes. For these reasons, the STS system can accommodate 
many needs presented by user’s with cognitive differences. However, in order to 
utilize the system most effectively, users should have an understanding of picture/object 
associations. 

I 

For the individual operating the system using the scanning mode, an understanding of 
cause/effect using a switch is desired. 

Speech In the direct selection mode, the user is not required to produce intelligible 
speech sounds. However, the sounds must be used consistently and repeatable on 
demand. Some sound production variation can be tolerated by the system, however, as 
the number of messages in the system increases, the distinctness of the sound is of 
greater importance. 

In the scanning mode, the system will recognize any sound produced by the user and 
thereby activate the appropriate item on the display. It is not necessary for the speech 
sound to be consistently produced each time. 

Single Switch Control In the scanning mode, the STS system can be controlled using 
any switch with a 1/8" stereo plug. Thus, for individuals for whom the STS system is 
appropriate but require use of a single switch, this is possible. 

The decision to use a single switch, should be determined by the user with an 
occupational therapist or other professional who can most effectively determine range 
of motion and movement. These recommendations should be used in the selection of a 
switch as well as switch placement. 
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Microphone Placement A variety of microphones are available which can be used 
with the STS system. Some individuals prefer not to have a microphone attached to 
their clothing, mounted in front of their mouth etc. Microphones which best 
accommodate the needs of the user and their preference can be purchased from 
manufacturers who sell electronic equipment. Additional information is available in the 
Appendix. 

Vision/Hearing A determination of the user’s functional hearing and vision should be 
established. These abilities enable the user to 'monitor system activity. With regard to 
vision, the user must be able to see the display as well as visually track/ scan the display 
for the image of the desired item. This is particularly important for individuals using 
the system in the scanning mode. 

Team Decisions Regardin£ Candidacy As described above, the information 
necessary to determine the appropriateness of the system for an individual can be 
provided by a number of individuals. This includes the user, family, occupational 
therapist, physical therapist, speech/language pathologist, teacher etc. It is 
recommended that all these individuals participate in the evaluation and implementation 
of the system to ensure its effectiveness not only at the time of evaluation but in 
support of the system later. 
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SYSTEM DESCRIPTION 



The main features of the STS/Graphics system are as follows: 




Voice Input 

The system uses the Votan 2000 voice recognition circuit board and software routines. 
Each user can store voice templates for sets of up to 64 messages, consisting of sounds 
or word phrases of up to 8 seconds in duration. Multiple users can simultaneously 
store and access templates for their messages. The templates for each message can be 
entered into the system in 2 to 3 passes. For individuals for whom consistent sound 
production is not possible, the STS/Graphics system can be adapted to accept simple 
vocalizations as a means of selecting desired items from a visual scanning routine. 

Sound-to-Speech Translation and Expansion 

The STS/Graphics system incorporates an algorithm that translates any designed sound 
input, whether intelligible or not, into a specified speech output. For example, if a 
user’s vocalization for water approximates "wuh" and s/he only speaks in single- 
syllable vocalizations, the system can immediately output "Could I have a drink of 
water, please. " The speech output is digitized and generated by the Votan 2000 circuit 
board. Any single speech output can be up to 8 seconds in duration, thus permitting 
single words or complete sentences output. 

Environmental Control 

The STS/Graphics system incorporates X-10 technology. Any voice (or other) input 
can be linked to the activation and deactivation of any electrical device. A single voice 
input can be linked to any combination of spoken outputs and device activations. 

Graphics System Interface 

The STS/Graphics system is able to generate photographic-quality images of items, 
appliances or people in the user’s environment. Both the size of the image and the 
number of images appearing on the display can be customized for a user. 
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SOUND TO SPEECH/GRAPHICS SYSTEM REQUIREMENTS 



The following hardware and software components are necessary for operation of the 
Sound-to-Speech software. It should be noted that the equipment listed below is the 
minimum configuration required for software operation. Enhanced performance is 
obtained using a 386 processor and hard disk with additional storage capability. 



Hardware and Software Requirements 

1. IBM PC-AT 

2. 30 MB of Hard Disk Space 

3. Super VGA Monitor 

4. Paradise Professional Video Adaptor 

5. Votan 2000 SRB 

6. Image Capturing Device 

7. X10 Powerhouse with Serial connector and wall modules 

8. 640KB RAM 

9. 5 1/4" Floppy drive 

10. MS-DOS 3.30 

1 1 . Sound to Speech Software 

12. Image Capturing Device Driver 

13. Image manipulator (PC Paintbrush IV+ Recommended) 
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Software Configuration and Design 

The Sound-to-Speech/Graphics System permits communication displays to be created 
which are customized to best meet the needs of the user. Photographs are taken of 
familiar items in the user’s environment which can be incorporated into the displays. 
The system operator uses a scanner to place the image onto a floppy disk. It is then 
transferred to the display using the STS/Graphics software. 

Displays can be created of varying sizes ranging from 2x2toa5x5 matrix. The size 
of the display and the number of images used is dependent on the needs and abilities of 
the user. For each item chosen for the display, a spoken message and/or environmental 
control function can be selected. For example, a photograph of a tape player might be 
configured to produce the message "I like this music" as well as activating the tape 
recorder to play music. 



Image Capturing 

The Sound-to-Speech software is designed specifically to utilize high resolution, color 
graphics. Using an image capturing device such as a scanner or a camera, photographs 
of objects are reproduced into a PCX file format which is readable by the 
STS/Graphics software. 

After the photographs have been digitized it is necessary to size them for use in the 
matrices used by the Sound-to-Speech software. A "paint" style software package is 
used to size the image to the exact specifications of the Sound to Speech package. 



Creating Photo and Message Libraries 

Photo Library . Prior to operation of any of these software components, pictures of 
objects need to be digitized. It is recommended that a collection of photos are made to 
facilitate speedy operation of the software. Photo images may be stored according to 
specific topic categories and size. These images are stored on floppy disks for easy 
retrieval and use during display construction. 

Message Library . A second library should be created of audio output messages. 

These are also categorized into groups which are similar to the library of digitized 
pictures. ARC has compiled a audio message library for both male and female voices. 
These messages are stored on floppy disks for easy retrieval and use in constructing 
user’s displays. 
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MAJOR SOFTWARE COMPONENTS 



The Sound-to-Speech software package is made up of three major components: Setup, 
Speech Manager, and Sound-to-Speech (STS). Each of these components is described 
briefly in the following pages. 



Setup 

The Setup section of the STS/Graphics software is responsible for creating a working 
display for communication and environmental control. The Setup portion of the 
STS/Graphics software operates in two modes: create a page or edit an existing page. 

A page is a collection of images and computer controlled functions which correspond 
with those images. Those images are arranged on a matrix in rows and columns. The 
dimensions of a matrix, e.g. 4 by 4, defines the number of rows by the number of 
columns. A cell is one location in the matrix. 

Building a Pag e. The Setup section combines picture files, audio output messages, 
and/or environmental control commands into designated cells of the matrix or display 
and facilitates the execution of these commands simultaneously. When information has 
been entered for each designated cell in the matrix, the information is stored as a 
"page" file. The number of cells which are filled is determined by the needs of the 
user. 

Setup allows the teacher to define the page name, the matrix dimensions and to place 
the pictures and recordings from picture and audio libraries into the specified cell of the 
matrix being built. 

At this stage, the user may also define any environmental control functions that are to 
be included in the page which being created. If an X-10 function is selected from the 
menu a prompt is provided by the software requesting the user to enter information 
regarding the wall module code associated with the environmental control function. 

This information will be assigned to designated cells contained on the matrix. Once a 
page is selected for use, all specifications determined in the Setup portion of the 
software are active and page creation is complete. 
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Speech Manager 

Speech Manager provides the audio recording and system training functions for the 
speech recognition board (SRB). 

Audio Recordings . Speech Manager facilitates categorization of individual audio 
messages into categories appropriate to the user or setting. New categories and 
messages can quickly and easily be added, similarly unused categories can be deleted. 
Once a category has been defined, audio output messages may be added, re-recorded or 
deleted. For each audio output message an area is available to further describe the 
stored message. These descriptions may be added to, deleted or modified. Message 
descriptions for a category can be displayed on screen, or printed for future reference. 

System Training . Speech Manager processes and stores the utterances or words that 
are used to control the system through direct selection or scanning modes. For use in 
direct selection access, the system trainings generated with the user’s utterance are 
matched with the associated picture. Thus the sound "pa" may be associated with a 
picture of nail polish, and "ray" may be matched to a picture of a radio. For direct 
selection access, each new picture included in the matrix is represented by a different 
sound. New sounds may be added to the file or sounds that are not readily recognized 
by the system may be retrained. 

Utilizing the scanning access method a single sound is used to train the system. Once a 
single sound is recorded and learned by the system, the range of recognition may be 
customized to the user. 

Retraining the system may result when a user has a cold or other conditions which may 
change the quality of the "trained" utterance. In most cases trainings remain reliable for 
many months without alteration. 
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STS 

STS executes any of the pages created by Setup and generates activity reports regarding 
the operation of the software. As each cell is activated by the user, the actions 
corresponding to that cell such as audio playback and environmental control are 
activated. In addition, information regarding this activity is stored in a data file. 

These data files are used to generate reports of system activity and use. 
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OPERATING SOUND-TO-SPEECH 



To activate the Sound-to-Speech program from a DOS prompt, check to see that the 
computer is logged into the disk drive on which STS is resident. Normally, drive C 
If help in change drive assignment is necessary consult your DOS manual for 
instructions. Once the computer is logged into the correct drive type STS and press 
ENTER. The copyright screen shown opposite will be displayed, press any key to 
sequence to the next screen. 
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THE STUDENT MENU 



The Student Menu screen that is displayed opposite is the first menu to be presented to 
the user when the STS software is loaded into the computer. 

This menu allows information on new students to be added to the system, student 
information that is no longer required on the system to be deleted, and existing data to 
be modified. Additional to the add, delete and modify functions this menu allows a 
student to be selected for use in all other aspects of the program. 

To Add a Student move the cursor, using the arrow keys to that field, and depress 
ENTER. 
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Adding a Student 

1. To add a new student type a unique name in the Name field, and press ENTER. 
If a name that is already on record is typed into the Name field, then you will 
be prompted that the name is already in use and you will not be able to save the 
name if you attempt to leave this screen. 

2. Once a unique name has been enter then the cursor moves to the Age field, type 
the students age and depress ENTER. If the students age is unknown then this 
field can be left blank. 

3. Enter the students diagnosis, if it is known, or leave the field blank. 

4 & 5. The next two fields on this screen, Person 1 and Person 2 refer to the people 

assisting the student in using STS, they also can be left blank if the student has a 
variety of people assisting him/her. .. ( 

6. Finally, enter the location where the student is going to be using STS. This 
field as the previous five lines does not have to be filled in. 

If changes need to be made to the information that has just been entered, use the arrow 
keys to move to the line that requires altering and make the correction. 

Once all the fields have been filled in correctly then use the Exit and Save field of this 
screen which allows the information that has just been entered to be saved on the hard 
disk and the operator to be returned to the Student Menu so that other student 
information can be added or edited. 

If you wish to use the just entered student in other areas of operation then move the 
cursor to the Select Student field and depress ENTER. This will sequence to the STS 
Main Menu. 
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Deleting a Student 

If student information is to be deleted from the system then move the cursor to the 
Delete Student field of the Student Menu and press ENTER. The screen marked (B) on 
the opposite page will be displayed. 

Press ENTER to move to the Student List window and move the cursor to the student 
that is to be deleted, and press ENTER. 

A red window will open once you depress enter, and the specified student will be 
deleted. 

Editing a Students Information 

From the Student Menu select Edit Student Description to modify student information 
that has already been recorded. 

Once the Edit selection has been made a window displaying the list of students who are 
saved on the hard disk is opened. Using the arrow keys move the cursor to the student 
whose information is to be edited and press ENTER. 

The Edit Student screen will be displayed. This screen is like the Add Student screen. 
All entries can be changed by moving the cursor, using the arrow keys, to the line that 
requires editing and then typing the correction. 

Note that the name cannot be changed, the only method that can be used to change a 
name is to use the Delete Student function and remove the student entirely and then add 
the student as a completely new entry. This safe guard is necessary to prohibit 
duplicate names being entered into the computer and thus creating problems when data 
is to be retrieved. 

Once all corrections have been made to the students information press ENTER to save 
the edited data. This will automatically sequence the software back to the Student 
Menu. 

Selecting a Student 

To allow STS/Graphics software to function in all other aspects than manipulating 
student information, then a specific student must be identified and activated. Move the 
cursor to the Select Student option of the Student menu and depress ENTER to activate 
a specific student. 

A window will open displaying all students who are available for activation, move the 
cursor to the desired student and press ENTER. This will automatically select the 
defined student and sequence to the Main Menu. 

The Exit option on this menu allows a user to leave the STS/Graphics software and 
return to DOS. To utilize this function move the cursor to the Exit line of the menu 
and depress ENTER. 



179 



Page 14 of Section 2 



Sound-to-Speech 

Setup 

Instructions 



ISO 



SOUND-TO-SPEECH SOFTWARE DESCRIPTION 



Section 2 






Page 15 of Section 2 



SOUND-TO-SPEECH SOFTWARE DESCRIPTION 



Section 2 



THE STS MAIN MENU 

Once a student has been selected STS/Graphics is able to operate and the Main Menu is 
available. The Main Menu cannot be accessed unless a student has been select. 
Once the Main Menu has become available then four options are accessible. STS 
which is the student interactive portion of the software, Setup which allows 
configuration of ’pages’ that will be operated by the STS portion of the program and 
Reports which provide details of the actions taken during the operation of STS. 

The four areas of the main menu need to be operated in the correct sequence. Each 
section will be discussed to provide insight into its operation, and then the following 
section will explain the logical succession for the best operation of the STS software. 
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SETUP 

From the Main Menu select Setup and depress ENTER. 

The program will sequence to a screen which offers three options; Start a Page, Edit a 
Page, or Exit. Exit returns you to the Main Menu. Start a.Page allows a new page to 
be created. Once this option has been selected the following instructions are displayed 
"type the name of the page you wish to construct [ ]. Enter a page name (8 

letters or number or combination of both) and press ENTER. The Scan Method screen 
is displayed once a page name has been entered. There are three access methods 
offered on this screen. Linear scan, direct select, and row/column scan. See page XX 
for a detailed description of each of these methods. Move the cursor to the desired 
field and press ENTER. 

If you do not wish to continue with construction of this page move to the Exit field 
which will return you to the Main Menu. . , 

l 

Once the scan method has been selected the dimension of the matrix which is to be 
constructed is requested: 

Enter the desired number and press ENTER. 

The final screen in the sequence for accessing the Setup mode is used to define the type 
of switch access that will be used with the specified page. Voice and Single switch are 
the options, select one and press ENTER. 

Once all of these selections have been made the Setup screen will be displayed. 
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Setup Screen Layout 

The setup screen has four major sections. In the bottom right of the screen is an 
outline of a matrix, this will be laid out with the number of cells which were defined 
prior to accessing this screen (1, 4, 9, 16, or 25). In the center of each cell in the 
matrix is a cursor symbol which is colored. In the top right of the screen is a color 
legend which defines the colors that these cursors can be displayed in. The color is 
associated with the status of a cell in the matrix. Green indicated that both a picture 
and an action have been defined for a cell, yellow reflects that only the picture or 
action have been designated, not both, and red indicates that the cell is empty. In one 
of the cells of the matrix the cursor will be blinking, this indicates the cell that is 
currently being configured and all functions both picture and actions that are defined at 
this time, will relate to that cell. 

The box at the bottom left of the screen displays information relative to the cell of the 
matrix which is currently being configured (i.e., has the blinking cursor). Picture 
name, message number, environmental control function, and action are all displayed if 
those functions have been selected. 
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Setup Screen Menu Bar 

The top left of the screen has a menu line which has two options, Picture and Action, 
these are activated by depressing P or A from the keyboard, respectively. When 
Picture is activated the software will look at the floppy disk in disk drive A and 
display, in a pull-down window, the names of the picture files stored on the diskette. 

If no diskette is resident in the drive at the time enter is pressed then a prompt to insert 
a diskette will be displayed. Once the list of picture files is displayed move the cursor 
to the desired picture and press ENTER. Note that in the lower left box the name of 
the selected picture file will be displayed. 

Action opens a pull-down window that displays a list of action options. This window is 
illustrated opposite. Each of these options will activate a secondary window and in 
some cases multiple windows will be opened. 

Playback opens a second window as shown opposite. The information required to fill 
in this window relates to the recorded speech output messages (see section XX for more 
details). The first field, Message it should be filled in with the number assigned to the 
desired message, this number is found in the right column of the Message Report. 

Enter the number and press ENTER. The cursor will automatically sequence to the 
next field, Source Drive this should be filled in with the drive name where the selected 
message file is stored. Once both of these field have been filled in correctly move to 
the Exit & Save field and depress ENTER. Selecting X-10 from the Action Menu 
opens an additional window as illustrated. The Code field allows the identification of 
the X-10 wall module number. In this version of STS the house codes, available on X- 
10 wall modules is configures by the computer as module A. Enter the desired house 
code number and press ENTER this will move the cursor to the Action field, if the 
house code that has been entered needs correcting move back to the Code field and 
make the correction, if the number is correct depress ENTER in the Action field to 
progress. Once enter has been pressed the following window will open: 

This window provides all of the options available for the specified X-10 wall module. 
On will always turn an appliance on and therefore there should be a second option in 
the matrix which will provide for the appliance to be turned off, this should be 
addressed when designing the configuration of each page. Off will only turn appliances 
off and as described in the previous sentence should have a companion cell in the 
matrix to turn the appliance on. Toggle will combine these functions in one cell. The 
first time a cell is accessed by a user, when is defined to toggle, the appliance will turn 
on. The second time the cell is accessed by the user the appliance is turned off. One 
device at a time allows only one electronic appliance to be activated at any time. This 
field activated another window as follows: 

Fill in the amount of time that an appliance is required to remain on before 
automatically turning off. Thus if One device at a time is selected the device turns on, 
remains on for the given amount of time and then turns off. During the time that this 
device is on no other device can be activated. 

From the Action Menu if X-10 & Playback is selected then the previously described 
sequences will be sequence consecutively, first the Playback portion and then the X-10 
portion. 
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The IR option from the Action Menu provides access to electronic components that 
operate with an infrared remote control. 

The Paging field of the Action Menu allows a cell in the matrix to be designated as an 
access cell to another page. If a cell is identified as a Paging cell then when it is 
accessed by the user the display on the monitor is replaced by a secondary page. On all 
secondary pages a "return” cell has to be designated, this cell allows the user to return 
to the original screen. 

When the Paging field is selected a window opens will the question Name of Page to 

page to . Enter the name of the secondary page and press ENTER. This will 

return you to the Setup screen. When the page that is being configured is complete and 
is being recorded to disk then the user is prompted to designate the "return" cell on the 
linked page. A window will open during the ’save’ sequence which prompts 

You must choose the return cell for page (name of designated secondary page) 

If the secondary page exists on the hard disk then it is loaded into the Setup software 
and the matrix cursor is located in the desired cell. 

If the secondary page has not yet been created then is must be constructed following the 
Create a Page process and the return cell must be identified. It is not necessary at this 
time to complete the page, but it must be created. If multiple secondary pages are 
designated from a page then the previously described process will be repeated until all 
return cells have been designated. 

If Paging & Playback is selected the cell and page are identified followed by the 
Playback sequence. 

This picture and action selection process should be followed until all the cells that are 
required to be filled are complete. Press ESCAPE to leave the Setup screen. This will 
automatically save the configuration that has been designated in a file of the name 
designated at the beginning of the Setup procedure. 
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Modifying Existing Pages 

The edit a page option from this screen allows modifications to be made to an existing 
page. Move the cursor to this field and depress ENTER to activate the Edit mode. All 
of the same sequence for editing is followed as is described in creating a new page. 

For more details refer to the previous section of this document. 
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SPEECH MANAGER 



Speech Manager is accessed through the Main Menu. Once it has been selected the 
Main Menu is replaced by the Speech Manager Selection Menu, as illustrated opposite. 
There are two options available through Speech Manager, Recognition Function and 
Training Function. When either of the functions are active they provide information in 
a windowing format. A window will open and present options or actions and as these 
options are chosen then second and third window open over the original menu. Follow 
the instructions to complete all of the functions associated with these windows. 

Recognition Function 

This function allows the software to be "trained" to recognize a specific students sounds 
which are used to activate, or act as a switch, for the software when it is being operated 
through the STS module. 



Training Function \ 

The Training Function is used to input and record the messages that the STS software 
actives when speech output is requested by the student. 
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Speech Manager Recognition Function 

When Recognition Function is selected from the Speech Manager Menu then the 
screen, illustrated opposite is displayed. There are two column displayed on the 
screen, A, which lists all of the pictures selected for the student, and B, which is a list 
of the sounds used by the student in association with each picture. 

The Picture list is compiled automatically by the software. Each time a unique picture 
is selected during Setup to be used in a page file then the name of the new picture is 
added to the Picture list. If the picture file has already been used then it will not be 
added to the list a second time. For Direct Selection each picture must have an 
associated sound. For scanning only one sound needs to be defined. 

Four functions can be activated from this screen: Train a Sound, Delete a Training, 
Help, Modify a Synonym. Each of these functions is activated using the ALT key and 
a second key simultaneously. 

ALT T activated Train a Sound 
ALT D activated Delete a Training 
ALT H activated the Help screen 
ALT M activated Modify a Synonym 



Training a Sound 

To train a sound move the cursor to the desired picture name and press the ALT and T 
keys at the same time. A window will open in the center of the screen which requests: 

Insert the disk containing "Name of Picture" in drive A and press ENTER" 

If you do not wish to continue press the ESC key and control will be returned to the 
Speech Manager Menu. 

To continue with the training sequence place the appropriate disk in disk drive A and 
press ENTER. The will be a brief pause as the system reads the file from the disk, 
once this is complete the picture will be displayed on the monitor. 

At the top left of the monitor will be a one line window which provides instructions: 

"Press Enter to record" 

When the student is ready to vocalize press the ENTER key, a beep will sound and the 
system will wait for a vocalization. The first sound uttered by the student will be 
recorded and a message will be displayed: 

"Save recording Y/N" 

If the user feels that the students utterance was appropriate then Yes should be 
answered to this prompt. If the student produced a sound other than the desired 
utterance then No should be entered and the sequence repeated. Once Yes has been 
enter then the complete record process is repeated a second time. Once again if the 
utterance is not the sound expected to be associated with the displayed picture then the 
record sequence can be repeated. After two complete cycles through the record process 



Page 29 of Section 2 



SOUND-TO-SPEECH SOFTWARE DESCRIPTION 



Section 2 



then the system automatically chains to a test procedure for the new recordings. The 
user is prompted: 

"Test the recognition Y/N". If Yes is answered then a beep is emitted and the prompt: 

"Say it " is displayed. Once the student utters a sound the system compares the 

sound to the two new recordings. If the test sounds is very similar to the recordings 
then the prompt: 

"Good recognitions" is displayed. If the similarity is poor the "Bad recognition" is 
displayed. Once either of these are displayed then the system presents the option: 
"Retested the recognition Y/N". Answering No to this prompt will return control to 
the Speech Manager Menu, answering Yes to the prompt will start the test sequence 
over again. 

This training sequence can be repeated for all of the pictures listed. If a training is 
omitted then when that specific picture is displayed in a page then it will not be able to 
be accessed. 



Deleting a Training 

Once the training sequence has been completed then the sound can be used indefinitely. 
Over a period of several weeks or months the students production of the specific sound 
can change, or if the student has a cold then the sound might not be recognized by the 
system. In the event of this occurring then the recording can be deleted and retrained. 
If a picture is no longer being used by a student then the sound should be deleted. 

To Delete a Training move the cursor to the name of the picture that is associated with 
the training. Press the ALT and D keys at the same time. A prompt will be displayed 
asking "Are you sure you wish to delete this training Y/N". If Yes is answered then 
the delete process is completed. If No is answered then control is returned to the 
Speech Manager Menu. 

Speech Manager Help 

If during operation of the Speech Manager program assistance is needed in how to 
operate the software then press ATL and H at the same time and instructions will be 
displayed on the screen. Page up and page down will sequence through the help 
screens ESC will exit the Help screen. 

Modify a Synonym 

The synonym list to the right of the screen is used to identify the sounds produced by 
the student for each picture. To enter or change a synonym press ALT and M at the 
same time while the cursor is located on the cell that is to be added/changed. Type in 
the new sound and press ENTER. The new sounds is recorded and will be displayed 
each time this screen is viewed, until it is deleted or modified. 
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Speech Manager Training Function 



The Training Function provides a method of entering messages in both written and 
audio form. Messages can be divided into categories for easier access and storage and 
can be printed in this form. Once the Training Function is selected from the Speech 
Manager Menu then a window opens which presents five options relating to Categories, 
as illustrated opposite. 

It is not essential to divide messages into groups but it is easier to keep them sorted if 
this function is used. There must be a least one category defined for each user to allow 
the message recording process to continue. 

Add a Category 

When Add a Category is selected from the Category window then a second window 
opens which requests: "Enter Category Name [ ]". Enter a name of up to 8 

alpha numeric characters. It is advised that the category name should relate to the 
types of message which it will contain, e.g., food, games, needs. Once the name has 
been type then press ENTER which records the name and closes the window. 

Select a Category 

From the Category window if Select a Category is chosen then a window opens which 
displays all of the Categories that are presently stored in the students information. 

Move the cursor to the desired Category name and press ENTER. Another window 
opens which displays the Message Menu. 

Modify and Delete a Category 

The Modify and Delete functions from this window are similar in operational procedure 
and therefore will be described at the same time. Once either is selected from the 
Category window the list of existing categories is displayed in another window. Move 
the cursor to the category that is to be modified or deleted and then press ENTER. If 
the category is to be modified then the name will be displayed in a window which will 
allow the name to be edited by typing or deleting characters. Once the modifications 
have be completed then ENTER is pressed and the changes will be recorded and the 
window closed. If the selected category is to be deleted then the option "Delete Y/N" 
is presented if Yes is selected then the deletion is completed, if No is chosen then 
control is returned to the Category window. 

Exit 

Exit returns control to the Speech Manager Menu. To activate this function move the 
cursor to the Exit field and press ENTER. 
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Speech Manager Message Window 

Once a Category has been selected then the Message Window is accessed. This 
window provides 8 options which are illustrated on the opposite page and will be 
described in the following section. 

It should be noted that messages are stored in two forms. As text and as an audio 
message. The options that are provided at times relate to one or other and sometimes 
both types of messages. 

Add a Message 

When Add a Message is selected from this window then a secondary window open with 
the instruction, "Type the message" Enter the text of the message which will be 
recorded for playback during STS operation. Once the text is entered press ENTER the 
window is replace with another window which instructs, "Press ENTER when ready to 
record". Check to see that the person who is going to record the message is correctly 
positioned in front of the microphone and press ENTER. The person recording the 
message should begin talking as soon as ENTER is pressed. The software "listens" for 
the silence after the message and as soon as the speech halts the system ends it 

recording process and displays a message saying " Got it". The dots appear as 

the speech occurs. The window now displays a message "Press ENTER for playback", 
this initiates playback of the message which has just been recorded and gives the user 
the opportunity to hear what the message sounds like. Once the message has been 
played back then the option "Re-record the message Y/N" is displayed. Answering 
Yes at this prompt will return the user to the beginning of the record sequence. 
Answering No to this question returns control to the Message Menu. 

Modify a Message 

When Modify a Message is selected from the Message window another window is 
opened which displays a list of the messages that have already been recorded. To the 
right of the text a number is displayed. This is a number which is automatically 
assigned to each message by the software and is used during the page setup procedure 
to define which message will be played back. If more messages are stored than can be 
displayed in the window then use the down arrow to scroll to the lower messages. 

Once the desired message is identified by the cursor press ENTER. A window will 
open with the message displayed in it. Use the arrow keys to move to the part of the 
message that is to be altered and type in the correction. Once the text is corrected then 
the procedure is the same as Adding a Message. 

Retrain a Message 

Selecting Retrain a Message from the Message window activates the list of messages, 
select the message that is to be retrained from the message list by locating the cursor on 
the text and pressing ENTER. The software will sequence to the record function that is 
explained in the Add a Message section above. 
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Delete a Message 

If Delete a Message is selected from the Message window then the list of messages is 
displayed. Move the cursor to the message that is to be deleted and press ENTER. 

The prompt "Are you sure you wish to delete this message Y/N" will be displayed, if 
Yes is selected then the message is deleted and control is returned to the Message 
window. If No is selected then no action takes place and control is returned to the 
Message window. 

Edit Message Text 

When Edit Message Text is selected from options in the Message window then the list 
of messages is displayed. Select the message that is to be altered by locating the cursor 
on it and pressing ENTER. The selected message will be displayed in a window and 
alterations can be made to the text by retyping the message, using the delete or arrow 
keys or a combination of all of these keys. Once the message is modified press 
ENTER to record the altered text. Control will be returned to the Message window. 

Play a Message 

As with Delete and Edit a message once Play a Message is select from the Message 
window options then the message list is displayed, select the message from the list that 
is to be played and press ENTER. The audio message associated with the selected text 
will be played and control will be returned to the Message window. 

Print Messages 

When Print Messages is selected from the Message window then a second window 
opens asking if the printer is ready, check to see that the printer is connected to the 
correct port on your computer and press Y. A list of all message stored for the selected 
student will be printed by category. See the illustration on the opposite page. 

Exit 

Exit returns the software to the Speech Manager Menu. 
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STUDENT SUMMARY 



Date/Time: Wed Jan 29 09:14:37 
Name: Maggie Sauer 



< 



Age: - 
Diagnosis 
Assistant 
Assistant 



V. 



Location: 




1992 



The Arc 

SOUND-TO-SPEECH 



DISPLAY CONFIGURATION 

f Access Metnod: Direct Select ! \ 
Page: DIRECT 
Matrix Size: 3x3 
Initial Recognition Distance: 50 
Matrix Color: Black 
Scan Color: Green 



CELL INFORMATION 



n. 



< 



Cell 1: 

Synonym: 

Cell 2: LIPGLOSS 
Synonym: LIPGLOSS 

Cell 3: 

Synonym: 

Cell 4: REDPOL 
Synonym: REDPOL 

Cell 5: 

Synonym: 

Cell 6: LOTION 
Synonym: LOTION 

Cell 7: 

Synonym: STIC 

Cell 8: STICKER 
Synonym: STIC 



V_ 



Cell 9: 
Synonym: STIC 



Message: 0 
Message: 3 

Message: 0 
Message: 1 
Message: 0 
Message: 2 
Message: 0 
Message: 4 
Message: 0 
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DATA COLLECTION AND ANALYSIS 



A chronological report is generated of all system activity is generated whenever the 
system is used. Subsidiary reports are generated from the chronological report 
detailing specific activities such as environmental control and audio playback. The 
subsidiary reports include an analysis of these activities. 



Student Summary 

The Student Summary is divided into three main sections; identifying information, 
display configuration and cell information. 

The identifying information section (a) is provided to label training dates, participants 
and location. Since user performance may vary with each of these variables, this 
information is important complete each time the system is used particularly in the early 
stages of system training. 

The display configuration section (b) provides a summary of how the system was setup 
for the user. It describes the access method, matrix size, recognition level, matrix 
color and scan box color. This information is helpful in comparing user performance 
as these variables are changed as well as referencing a change in performance if the 
variables are unintentionally changed during the training process. 

The cell information section (c) provides a written description of the contents of each 
area or cell on the display. This includes the description of the picture in the cell as 
well as the message which will be spoken by the computer when the cell is activated by 
the user. 
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SOUND-TO-SPEECH 
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Date/Time: Wed Jan 29 09:14:37 1992 
Name: Maggie Sauer Age: - 
Diagnosis: - Assistant .1: - Assistant 2: 
Location- 



Page: DIRECT 

Access Method: Direct Select ii Matrix Size: 3 x 3 
Initial Recognition Distance: 50 
Matrix Color: Black Scan Color: Green Beep: On 



^Eve 



nt 
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BOX 8 


T ime 


Cell Name 


Device 


2 


09:14:37 


LIPGLOSS 


Playback 


8 


09: 14:41 


STICKER 


Playback 


4 


09:14:46 


REDPOL 


Playback 


6 


09:14:50 


LOTION 


Playback 


4 


09:14:54 


REDPOL 


Playback 


2 


09:15:11 


LIPGLOSS 


Playback 


8 


09:15:14 


STICKER 


Playback 


4 


09:15:25 


REDPOL 


Playback 


6 


09:15:30 


LOTION 


Playback 


8 


09:15:33 


STICKER 


Playback 


4 


09:15:38 


REDPOL 


Playback 


2 


09:15:43 


LIPGLOSS 


Playback 



Function 

I want to put on lipgloss 
Stickers are sexy 
I love bright red nails 
My hands are dry, I need lotion 
I love bright red nails 
I want to put on lipgloss 
Stickers are sexy 
I love bright red nails 
My hands are dry, I need lotion 
Stickers are sexy 
I love bright red nails 
I want to put on lipgloss 
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CHRONOLOGICAL REPORT 



The chronological report is divided into three main sections; identifying information, 
display configuration and event. 



The identifying information (a) and display configuration (b) information is identical to 
the information provided on the Student Summary page. Duplication of the 
information on the chronological report will assist staff in matching the student 
summaries and chronological reports for future reference. 



The event section (c) enables the facilitator to track item selection and system use 
across time. This is information is useful in determining: 



a. Preferences 

b. Functionality and use of items on the display 

c. Time required by the user to make selections 

d. Errors produced during training sessions. 



This information will assist the facilitator in making changes in system options, the 
actual items available to the user on a display, or the number of items provided for the 
user. 

Additional information regarding data collection and analysis has been provided in 
Chapter 8 - Case Examples. 
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STS 

When Student Information, Messages and Recognition Trainings, and Page Setup have 
been completed the software is ready for operation. Select STS from the Main Menu 
to run the interactive portion of this software. 

Once STS has been selected then a list of pages is displayed. Select the page that is to 
be operated, from the list by positioning the cursor on the desired page name and 
pressing ENTER. 

The screen will clear and then the page will be displayed. Depending upon the 
configuration in Setup the page will function differently. If access was designated as 
through scanning then once the page is displayed the scan box will start to move 
through the matrix in the predefined manner, (linear, or row/column). The scan box 
can be halted by the student producing an utterance. 

If access was through Direct Selection then the page will be displayed and be 
immediately in a active mode. It will wait for sounds and once a sound is produced 
then the system will try to match the sound to a recorded utterance. If the system finds 
a similar sound then it will execute the appropriate functions that were defined to 
activate with the match of that sound. 

Function Keys 

There are nine keys that can change how the STS software reacted during operation: 

FI Increases the speed at which the scan box moves through the matrix. 

F2 Decreases the speed at which the scan box moves through the matrix. 

It should be noted that each time either key is pressed the scan box must move 
one cell before the key stroke is acted on, e.g., if FI is pressed twice then the 
scan box will move two cells before the box moves faster. 

F3 Acts as a toggle and turns the beep, which is associated with the scan box 
| movement on or off. 

F4 Changes the matrix color. Each time F4 is pressed the matrix color changes 
through green, blue, red, yellow, and black. 

F5 Changes the color of the scan box through the same colors as the matrix. The 
scan box colors sequence independently from the matrix color changes. 

F6 F6 controls the start position of the scan box. It can either always return to the 
top left cell in the matrix when the scan is halted, or continue on to the next cell 
in the matrix after the scan is halted. 
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F7 



F7 provides a method of triggering message output from the keyboard for a 
specific cell. This function is used during the early stages of training to provide 
feedback to the student even when their vocalizations are not accurate. 



+ The + key can be pressed repeatedly to increase the recognition of the system 
for utterances. If the recognition is widened then a greater variance in the 
utterances produced by a student will be match to the trained sounds in the 
system. 

The - key decreases the recognition rate of the system and makes it less able to 
match utterances to the trained sounds. As a student becomes more able to 
produce sounds consistently then the recognition can be decreased. This allows 
more sounds to be used with the system without them being miss matched. 






210 



Page 41 of Software Description 3/1/93 



Section 3 

Initial Considerations 



SOUND-TO-SPEECH INITIAL CONSIDERATIONS 



Section 3 




212 

O 

ERIC 



Page 1 of Section 3 



SOUND-TO-SPEECH INITIAL CONSIDERATIONS 



Section 3 



INITIAL CONSIDERATIONS 



Setting up the STS System 

There are some basic considerations which can enhance the operation of the system by 
the user. These include monitor height, microphone placement, and switch placement. 

Monitor height. The monitor height should be adjusted to a comfortable level for the 
user. The user should be able to scan the display without unnecessary vertical or 
horizontal head movement. 

Microphone placement . As discussed in Chapter 1, there are a variety of 
microphones which can be used by the STS system. Individuals assisting the user with 
system setup and microphone selection should take care to read manufacturer 
instructions regarding placement. These recommendations should be closely followed 
to ensure effective system operation. If a microphone is improperly placed, it can 
dramatically effect the ability of the system to recognize the user’s speech sounds. 

Switch placement. Appropriate switch selection and placement are critical to the user 
controlling the STS system in this way. For many individuals with physical limitations 
which require them to use a switch, careful switch placement is the critical difference 
between independence and dependence. Evaluation of the individual’s fine and gross 
motor skills is necessary to effectively determine placement. Information can be 
obtained from an occupational therapist assisting the user and/or caregiver. 



CUSTOMIZING DEVICE OPERATION FOR THE USER 



The STS system is designed to adapt to the needs of the user. These features were 
discussed in Chapter 1 but will be discussed here with regard to strategies for 
implementation. 

Choices/Selection of Photos for the Display 

In order to organize the photos which will be used for constructing the user’s 
communication displays, it is recommended that caregivers generate a list of 
objects/activities/people the user communicates about. After this list has been 
constructed, it will be easier for the caregiver and/or user to group similar items 
together. These items can then be grouped together on one display to create a topic 
board or categorically related group of items or messages. Most importantly, however, 
is the selection of items which are meaningful to the user (Please refer to the appendix 
for additional information). 

Research has shown that for persons with cognitive impairment, the use of items from 
their natural environment as well as training which occurs in these settings is most 
effectively generalized. The effectiveness of the STS system comes from the capability 
to implement photographic quality images of items/people in the user’s natural 
environment as well as the user’s own speech sounds. 
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LINEAR SCAN - The scan box moves from box to box sequentially 



Nr 










V 






















— \ 


V 






NK 


/ 


/ 


> 


7 



ROVV/COLUMN SCAN - The scan box moves between rows until it is halted by the 
user on a specific row. Once the row has been selected the scan box then moves across 
the row cell by cell until the user activates a switch that causes the scan box to stop 
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Customizing the Display for the User 

There are several options available which can enhance user operation. These include 
scan speed, recognition level, matrix/scan box color, and beep/signal. Each of these 
features may be adjusted from the keyboard while the STS system is being operated by 
the user. This permits the customization or "fine tuning" of the system as needed. The 
following keys on the keyboard can be used to change the systems operation: 



§cnn speed The rate at which the scanning box moves across the display in either the 
linear or row/column scanning mode is controlled by: 

FI which increases the scan speed 
F2 which decreases the scan speed 

Examples of each scan type have been illustrated opposite. 



Recognition Level This refers to how closely the user’s speech sounds must be 
produced, in comparison to the trained utterances, to be recognized by the system. 

+ Key expands the recognition level 
- Key narrows the recognition level 



Matrix Color Matrix colors include yellow, red, blue, black and green and may be 
changed by repeatedly activating the F4 key . 

F4 toggles the matrix colors 

Scan Box Color Scan box colors include yellow, red, blue, black and green and may 
be changed by repeatedly activating the F5 key. 

F5 toggles the scan box colors 

Beep/Signal The beep indicates scanning box movement on the display. It can assist 
the user in more effectively attending to the display. 

F3 toggles the beep on/off during the scanning mode 



The user should be exposed to each of these features and permitted to use them in a 
variety of combinations. Information should be recorded by support personnel or 
family members to determine which settings most effectively enhance user 
performance. 
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Speech Sound 
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Photograph 1 




































































-- 































216 



Page 5 of Section 3 



SOUND-TO-SPEECH INITIAL CONSIDERATIONS 



Section 3 



Additionally, the user’s sound for the item or activity should be recorded with the 
photographic image of the item on the display. The worksheet provided opposite 
permits the individual assisting the user in setting up the system to organize these 
speech sounds, images and accompanying messages. 



Examples of system operation for scanning as well as direct selection using voice 
control are contained in the Chapter 8-Case Examples. Examples of the system settings 
and user performance are provided. 
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t 

TRAINING CONSIDERATIONS 



In Chapter 3, some of the initial considerations for system use and customization were 
addressed. This chapter will discuss some of the training considerations related to use 
of the direct selection or single switch scanning mode. 

Direct Selection- Voice Control 

In order for a user to effectively control the STS system through direct selection or 
voice control , they should be able to produce a variety of sounds. These sounds may 
be unintelligible, yet, must be able to be repeated consistently. This consistency is 
necessary for the STS system to recognize the user. 

In order to begin making these determinations, interviews should be conducted with 
parents, caregivers, and other support staff regarding the user’s speech and repertoire 
of speech sounds. Many times, these individuals can identify speech sounds which may 
be used in reference to specific activities, events, objects or people. These sounds 
become the initial training point for the STS system. Particularly for individuals with 
cognitive impairment, structuring these training activities in this way can be critical for 
the following reasons: 

a. Enables the user to implement a speech sound they presently use as a 
meaningful message which will be provided by the computer. 

b. Utilizing their own speech sound rather than teaching sounds, the user simply 
has to learn that they are controlling the system with their voice rather than a 
sound which has been selected by others. 

c. Cognitively this task is easier because the user has already developed an 
association between the speech sound and the activity or event. 

In Chapter 3, creating a picture library using a vocabulary list was'recommended to 
customize the system to the user’s needs. Creating a message library which describes 
the user’s speech sounds and their meaning can be used to train the user with the 
appropriate communication display. Furthermore, these speech sounds can be used to 
train the system to recognize them and thereby "translate" these sounds into messages 
spoken aloud by the computer. In order to document these speech sounds and their 
associated referents, the table below has been provided as an example of how this may 
be organized. In this way, a list may be developed of the user’s speech sounds, their 
referent activities, objects or people and the intended message. The lists will enable 
personal care assistants, parents, and teachers to organize this information and place it 
in the STS system. 
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Training for the STS System should begin with very simple activities to give the user 
confidence and success in controlling the system. Additionally, it is best to begin with 
activities the user will successfully complete and gradually increase the complexity of 
the activity. For example, the user may begin with one item on the display. As the 
user gains control over the system with single items, additional items may be added to 
the display. 

It may be necessary to adjust the user’s recognition levels for the system as items are 
added. Following the training procedures described in Chapter 2, the system should be 
trained to recognize the user’s speech sounds to insure reliable system performance. It 
is recommended that the user "test" each item on the display to insure maximum 
response by the system. The system should be "retrained" for any item which the 
system fails to adequately recognize. 



Single switch/ Voice 

Initial Considerations Seating/positioning, switch selection and switch selection are 
critical ingredients to the effective use and implementation of the STS system. The 
appropriate adjustments and combination of these three factors provide the foundation 
for optimal system use. If additional assistance is required to address any of these 
areas, a physical, occupational, and or speech/language pathologist may be consulted. 



During the initial stages of training it may be necessary to provide the user with 
opportunties to practice using their switch and improve their control in operating it. A 
variety of opportunities may be provided outside the STS system to assist them in 
developing these skills. Examples of these activities may include controlling small 
household appliances, simple computer games, battery operated toys etc. For the user 
who does not have a range of speech sounds but prefer to use their voice to activate the 
system, voice activated switches are available and may be used in the manner described 
above. These voice activated switches are available from a variety of manufacturers 
and range in price and effectiveness. Additional information has been placed in the 
appendix Which describes switches, switch mounting hardware, and computer software 
programs which may be used for some of these activities. 

When using a switch or the user’s voice to control the scanning display on the STS 
system, a hierarchy of tasks should be provided. It is not necessary to follow these 
steps in order, however, they are provided as an outline of sequenced training steps 
which may be helpful to begin to gradually develop skills to enable them to effectively 
control the STS system. It should be noted that the suggested training strategies are not 
unique to the STS System. These strategies may be applied more generally to training 
controlled switch use and scanning. They are provided here to assist parents, 
caregivers and teachers in the use and training of the STS system specifically. 

Training for the STS System should begin with very simple activities to give the user 
confidence and success in controlling the system. Additionally, it is best to begin with 
activities the user will successfully complete and gradually increase the complexity of 
the activity. For example, the user may begin with one item on the display. By 
activating their switch, the STS system would then speak the accompanying message 
and/or turn on the accompanying appliance. This activity will give the user the 
opportunity to learn that their switch activation has caused something to happen on the 
STS system. When the user understands this relationship between the switch and the 
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item on the STS system, it is possible to move on to a simple scanning activity using 
one item on the display. 

At this time, the system should be further customized for the user. The options for 
customization of the STS system were discussed in Chapter 3. They include: 

scan speed 
recognition level 
matrix color 
scan box color 
beep/signal 

A variety of settings should be used to determine the user’s optimal performance for 
each setting. With regard to scan speed, the user may begin with a speed setting of 5 
and adjust the setting depending on the performance of the user. Each of the settings 
should be used in this way and adjusted as necessary. It should be noted that user’s 
performance may vary across time due to fatigue, attending behavior etc. These 
settings should be adjusted for the user’s performance accordingly. 

After adjusting settings for the user, a simple scanning task may be introduced. A 
single item in the 2x2 matrix may be presented. The user may be given opportunties to 
practice using the single item in a 2x2 matrix in order to develop the visual scanning 
skills and timing necessary to activate the switch when the scanning box is located on 
the desired item. Initially, this should be done in the linear scanning mode. As the 
user’s proficiency with this task improves, additional items may be added to the 
display. Adding items to the user’s display is discussed in detail in Chapter 2 - STS 
Desciption and Operation. 

A user’s speed during the scanning process may be enhanced utilizing the row/column 
method of scanning. This is particularly useful for larger display sizes such as 3x3, 
4x4 and 5x5. However, it should be noted that the row/column scanning method 
requires more skill by the user. It requires the user to locate the desired item on the 
display and plan for the selection. This is done by activating the switch in the row of 
the desired item and a second switch activation as the scanning box advances to the 
item within the row. This requires additional cognitive planning as well as switch 
timing skills. The user should be given opportunities to practice these skills before 
they are asked to locate desired items for communication and/or environmental control 
purposes. 



Choices 

The user should be provided with choices which are meaningful and motivating to them 
in their environment. This includes home, school and community activities. It is 
important to include choices or opportunities for communication which the user wants 
to communicate about. Choices should be provided to the user on the STS system 
which provide these opportunities. 
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Display Configuration 

The number of choices offered to the user will depend on their abilities. This can vary 
particularly with regard to how the individual is controlling the system. Most 
importantly, it is important to begin training the system with one or two items. As the 
user’s proficiency with the system improves, the number of items should be increased 
accordingly. 

Additionally, as discussed previously, it may be necessary to adjust the scan speed, 
matrix color, etc. to best accommodate the needs of the user. 



Additional information has been placed in the appendix of this training manual which 
further describes the construction of choice-making opportunities, training strategies 
and constructing topic boards. Topic boards are categorically specific communication 
boards which resemble the choice-boards constructed for the STS system. 




Page 6 of Section 4 



Section 5 
Case Examples 



225 



SOUND-TO-SPEECH CASE EXAMPLES 



Section 5 




FIELD TESTING THE SOUND-TO-SPEECH SYSTEM 



Subjects were referred by teachers, therapists and special education directors in the 
cooperating schools. For the evaluation phase of this project 24 subjects (12 male and 
12 female) were evaluated ranging in age from 4-21 years. Diagnosis for individuals 
evaluated, included cerebral palsy, autism, encephalitis, Down’s syndrome, and renal 
dysfunction. Subjects varied in range of mental retardation from mild to profound. Of 
the 24 subjects evaluated, 14 were ambulatory and 10 were non-ambulatory. Criteria 
for selection to the Sound-to-Speech Project included: 

o adequate vision and hearing 

o adequate positioning and seating 

o understanding of picture/object association 
o understand cause/effect relationships 

o possess rudimentary scanning skills (if appropriate) 

o speech is not functional as primary mode of communication 

Six subjects were selected for participation. Final subject information is described 
below: 

4 
2 

7-21 

cerebral palsy, encephalitis, mental retardation - mild to profound 
4 (Two walk with the assistance of a walker) 

2 






Female: 

Male: 

Age Range: 
Diagnosis: 
Ambulatory: 
Non-ambulatory: 



Case examples of system use were generated from these field testing activities and 
summarized for a user who controlled the system utilizing direct selection and a user 
who controlled the system utilizing a switch with scanning. 
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Single Switch Operation - Demmie 



Diagnosis - Moderate to severe mental retardation 
Age - 21 

Etiology - Origin of mental retardation unknown, restricted movement for walking, 
nonspeaking with the exception of a few speech sounds, speech was preferred method 
of communication regardless of intelligibility to others. 

History - Demmie was a young woman with moderate to severe mental retardation. 
She had been introduced to a variety of communication methods which included sign 
and communication boards. Each of these methods had been relatively unsuccessful 
because of Demmie’s preference to use her speech over all other methods of 
communication. She attended school and was limited in her ability to reliably 
communicate with her teachers, peers and family members. Demmie relied on their 
ability to "translate" her communication attempts across each of these environments. 



Recommendations for System Use - Although Demmie could not control the Sound- 
To-Speech system using individual speech sounds to represent each of her messages, 
she was able to use her voice as a switch to control the system through scanning. She 
was motivated and interested in this method of communication and using the system to 
assist her in accomplishing this. 

A list of Demmie’s preferred activities and communication messages were generated 
using the worksheets found in the appendix to this manual. Displays were prepared 
using these worksheets and pairing appropriate photographs for each of the items on the 
displays. Beginning with one item on the lxl display, Demmie gained more 
confidence in her control and operation of the system. As her skills improved additional 
items were added to her displays which consisted of beauty aids, leisure time activities 
and drawing or coloring activities. 

System Use - It was necessary to vary the scanning speed of the scanning box as 
Demmie’s skills improved. Additionally, two different colors were used to highlight 
the matrix color and the scanning box. This assisted Demmie in visually identifying 
the scanning box on the display as well as division on the display between her choices 
or pictures. 

The data collection tools available on the STS system assisted staff in comparing 
Demmie’s performance as these variables were changed. Furthermore, this assisted 
staff in making changes which improved her overall performance and control with the 
system. Staff were also able to discern which items were really functional and useful 
to Demmie on the displays. Thus, items which were not used or non-preferred could 
be replaced with more functional and motivating messages and pictures. 





Voice Control - Misty 

Diagnosis - mild to moderate mental retardation 
Age - 7 

Etiology - Unknown, nonspeaking, demonstrated the ability to produce a variety of 
speech sounds which she used to reference specific activities, objects and people in her 
environment. 

History - Misty was diagnosed as having mild to moderate mental retardation. Like 
Demmie, a variety of communication systems had been tried including sign and 
communication boards. Misty preferred to use her own voice to communicate and used 
this strategy most often with her parents at home. At school, Misty was described as 
"being quiet" and "not talking very much". 

Recommendations for System Use - Misty and her mother were asked to bring a 
variety of her favorite activities and toys with her to test the system. Her mother 
reported different speech sounds used by Misty with the six items provided for use with 
the system. This was confirmed by Misty’s production of different speech sounds each 
time the items were used during the introduction. Photographs of each of the items 
were scanned into the system and presented on the system display. The system was 
trained to recognize Misty’s speech sounds for each of the six items. Misty 
demonstrated the ability to produce each sound as the picture image of each item was 
presented. 

Displays were constructed for Misty using the information provided by her mother and 
school staff regarding Misty’s preferred activities and people. 

System Use - Misty began by using the system one item at a time and gaining 
confidence in using the system. As her skill and control improved items were 
gradually added to her displays one by one. 

The data collection tool for the STS system enabled staff to keep track of her 
performance with regard to errors, items used and not used and performance variance 
with different instructors. 

Overall, parents and school staff reported increased vocalization, interaction and 
improved speech intelligibility. 
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TOPIC BOARDS- ONLY THE BEGINNING 



Maggie Sauer M.S., CCC-S pi 
John C. Costello, M.S., CCC-SpI 

The following was modified based on information available through the Communication 
Aids and Systems Clinic, Madison, Wi. 

Mini boards 



1. Miniboards are small topic specific communication boards that are developed as 
part of an ongoing activity. They enable us to move beyond the simple topic or choice . 
boards which are often used in the eariy stages of augmentative communication 
development to enable the user to select a particular item or activity or topic. For 
example, topic boards might include choices of different foods for a meal or different 
toys for an activity or different activities. Topic boards do not enable the user or the 
message receiver, however, to make any comments about the activity or topic. 
Miniboards, however, provide a small evolving vocabulary centered around a 
particular topic or activity, designed to encourage conversation as an activity is 
underway, in some ways, miniboards are like the vocabulary selected for a language 
activity. Key vocabulary or language concepts are selected and introduced in the 
context of an ongoing activity. 



2. miniboards can sometimes be 
effective if they are constructed 



made up ahead cf time but generally are 
( on the go) as part of the activity. 



most 



3. In the initial stages, certain activities or favorite games might be selected as 
opportunities to use miniboards. Miniboarcs are not intended for use throughout the 
day. In a school setting, two or three of the daily activities might be selected for use 
with miniboards. (For example, opening discussion, snack time, a particular therapy 
activity). At home, meal time may be a good opportunity for miniboards. With some 
families, meals are too hectic to use miniboards and a snack time might be an 
alternative. Certain play activities or times during the day when family members can 
select a special activity might be prime times for miniboard usage. 



4. Miniboards may have only 4-6 symbols on them or they can become quite elaborate 
and include an extended vocabulary of 20 to 30 items. Here are some selected 
activities that might lend themselves to incorporation of miniboards in the activity. 
Examples of some of the vocabulary that might be appropriate for a given child or 
adult are included: 

ACTIVITY VOCABULARY 

Playing with a ball and a box ball, box, top or lid, in, out, all gone, where 

people's names, find, get’, drop. 

Building and knocking a tower over blocks, tower, numbers, sizes, shapes, up. 

hit. fall down 



Wind -Up figures 



names of peoce, figures names, stop, go, fall 
help, stand, v. id, on, off 



8owling game 



pins, ball, turn, stop, fall down, yean!, stand up 
too bad! 



Trucks, blocks 



truck, car, blc:k, more, go, stop, CRASH, hit, 
names of plaaes to drive iruc. - ^ or Cci iO (e.g. 
Mom, store, McDonald's, grancma’s). 



Miniboards could also be developed for washing a doll, buying things at the Siore, 
having a tea party., playing on the farm, speace ship, going up in a oailoon. In. tact, 
any activities that the child might enjoy could incorporate the use ot a miniboard, me 
to its successful use is selection of vocabulary that pertains to the situation and 
represents the , child’s interests. 

5 miniboards enable us to talk about a specific activity, to expand vocabulary 
available to a board user and to talk about specific aspects of the activity rather ti.ar. 
general comments. It is used in a turn taking mode both in play itself and in 
conversation. Both people are users of the communication board and active 
participants in the activity. 

6. It usually helps to have a matrix with blank spaces so that new items can be added 
on the spot. If the specific vocabulary items are needed regularly, however, it may 
be helpful to develop a specific drawing and select a standard arrangement for 
repeated usage, after a few trial constructions. 

7 If a set of blank matrixes are made up ahead cf time, it is easier to create new 
miniboards. The charts should be Organized in similar patterns on each chart so that 
the user learns that people’s names wifi be in the same area of the chart, action words 
in the same area, adjectives and emotional comments in their respective areas. 

8. Color coding, perhaps with a different ink for each category, wiil visually break up 
the display and perhaps speed up locating a certain item. 

9. Drawings used on miniboards should be bold and simple. Resource manuals for 
simple line drawings include: 

Mayer-Johnson 
P.O. 8ox1579 

Solana Beach, CA. 92075-1579 . 

• 8oardmaker - Macintosh software program used to create communication d.splays 

Picture Talk- Bruce Bollerud 
3113 Atwood Avenue 
Madison, Wi 53704 

Don Johnston Developmental Equipment, Inc. 232 



P.O. Box 63S, 1000 N. Rand Rd.. Bldg. 115. Ws.ronda. Illinois 500=i 
312-526-2682 

Standard Rebus Glossary 
American Guidance Service, Inc. 

Circle Pines, Minnesota 55014 

l 0 :" y °" have 1° c = bulary ilems «wi are used -speatedly on cirrerent minibosrcs 
those pictures could be xeroxed and the same drawing used. In addilien, it rasv 5 . 

useiul lo cover these displays with contact paper. Self-sealing ooeninos, or clear 
vinyl envelopes, or photo album pages are options as well. 

11 -he primary use of miniboards in these play activities is for conversation. Muc- 
of the learning of the symbols is carried on in the context of the activity through 
labe ling and modeling by the communication partner for the user. This techniques is 
similar to the commenting done in standard oral language therapy. Shod single were 
two end pernaps three word or picture combinations would be used at first to" 
comment on things as they happen. Occasional, questions might be addressed to the - 
communication board user and opportunities to make choices about who will p 0 

acMvit^ 9 n6Xt ’ What W! " b8 d ° ne ° r Wh ' Ch ' lem tC SelSCt Sn ° Uld be a pan of the ' 

rh : H ! l IS •I m ? 0rta - n u t t0 ail0w for silencss w 'lnin the conversation; oDDortunities for the 
child to initiate, ihe partners- should wait and allow this to happen. ' if the child or 

^nmoth 13 6S c / :Gmment verbdlly, through bodily gestures or reaction or by pointing to 
something on the communication board, the partner should reoeat that comment or 
their best interpretation of it, simultaneously pointing to the kev words on the 

communication miniboard, if the appropriate words are not there they should be 
drawn in quickly at that time. 

1 3 . Rather than requesting that a chid say phrase more completely, add additional 
symbols to their comment, by modelling. In addition, an alternate straieoy for 
encouraging expansion is by the user, is to point out a messaoe which is not 
comp.etely accurate as your interpretation and lock for clarification from the user. 

14 As the communication board user becomes mc-e adapt at usino the miniboards it 

' be nn P °rl' b e !° SeleCt vocabular Y i^ms used most frequently from all miniboards 
and consolidate them on a primary board. 



Donna DePape 

Communication Aids and Systems Clinic 
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ESTABLISHING REPRESENTATION THROUGH - OBJECTS 



Initially, NAME should receive instruction focusing on developing an object based 
communication program. Specifically, he should learn that he can use ari object as a " 
rccresentation in order to communicate a preference. Instruction with such a system will 
focus on the competencies necessary for a successful use of a more symbolic based 
communication system. The following is suggested: 



« 



1 . Identify highly reinforcing items and obtain duplicates of the objects. 

2. Identify mildly aversive or low interest items and obtain duplicates of the objects. 

3. Place one reinforcing item and . one lev/, interest item in large cubes, allowing NAME to 
see the items but not touch them. 

Suggestions for cubes include:'' ' - •- - • ■ ....... 

a. Clear photo cubes (5X8 size ) 

b. . Compact disk storage cases as demonstrated during today's meeting (found at 

Crate and Barrel). - 

c. Custom made display cases 

d. Clear plastic storage boxes available thrdugh many discount stores. 

A Ask NAME "What do you want?". 

5. Each time he reaches for/touches the box containing the desired object, he should 

immediately be provided with the duplicate of the object, thus demonstrating the 
representational value of the object in the box. 

6. Be sure to vary the side the desired item is placed on in order to avoid perseveration to 

. one side. . ' 




Costello, J.M., 19SS 
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FACILITATING OBJECT TO PICTURE ASSOCIATION 



Once NAME is clearly demonstrating representational skills and is consistent with makine 
selections, he should be provided with instruction focusing on a more abstract level or = 
representation. This will allow for an easier presentation of choices as well as a more 
functional means for a child who is independently ambulating to communicate in a varietv 
of settings. Three potential programs follow which have been successful with a varietv of 
people focusing on the goals of object to picture association. NAME’S rate of success 
with previous programs may dictate which of these three is tried first. 

1 . As outlined above, highly reinforcing objects as well as a mildly aversive object should 
be placed within a clear plastic cube (available in photo supply stores) in order to 
emphasize the representational value and three dimensional aspects of these items. 

NAME should be required to point to the cube containing the desired object. The 

location of the itcms-should be varied in order to avoid perseveration to one side and to 
ensure that NAME is attending to 'the item and not the location. Each-time NAME 
reaches for the cube, he should immediately be provided with the opportunity to interact 
with the requested item. The item provided, however, should be a duplicate of the item 
in the cube, thus preserving the representational nature of the cubes. Once NAME is 
consistently selecting the desired items, a photograph of the items should be suspended 
within the cubes, taking the place of the object. When NAME is consistently selecting 
the desired item, the photograph should be cut so that only the item is visible and the 
background is removed. This preserves the three dimensional representation althouah 
the stimulus is now two dimensional. The identical program described above should b 
carried out at this level. Finally, the photo should be removed from the cube and place 
on a flat board. 

2. Two photographs should be placed on the examiner’s side of a large sheet of 
plexiglass. NAME should be encouraged to look at these photocraohs while being 
asked "What do you want?’’. The actual object should be placed'behind the 
photograph, therefore, when NAME reaches for the object, he will touch the photo. 

He should then interact with a duplicate of the object behind the plexielass. In some 
cases, hand over hand modeling may be needed to teach NAME to point to the 
photograph. Following rrials at this level, the object should then be placed further out 
of sight (i.e., several inches back from photograph). When NAME reaches for the 
object, hand over hand modeling should be used to re-direct him to the photograph. 

The object should then be immediately presented. Through this association, NAME 
may learn to point to the photo to receive a desired item. The location of the photos on 
display should be varied in order to avoid perseveration to one side. Again, as NAME 
demonstrates success, the object should be moved further out of sight until it has finally 
been removed from behind the plexiglass. If NAME begins to consistently select the 
photo of the desired item while avoiding the mildly aversive rime, it will be clear that he 
is attending to the photograph and recognizes its representational value. 

3. Place the container of a favored food (e.g., fruited yogurt) which NAME recognizes on 
the examiner's side of a sheet of plexiglass. Each time NAME reaches for this" 
container, he should be provided with the food item (from a different container) 
immediately. The container should now be cut so that only the front half remains. This 
begins to reduce the three-dimensionality’ of the container. As NAME has success with 
this, the container should be cut down further until only the front panel remains. When 
NAME is able to consistently point to this front cover and avoid a less desirable 
choice, a photograph of the front cover should be presented. 
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FACILITATING INTENTIONAL COMMUNICATION BEHAVIOR 



In order to become a more effective communicator, NAME should be required to actively 
communicate his preferances. Thus. NAME should be provided with a choice 
(Antecedent), be expected to select a prcfcrancc (Behavior) and then be immediately 
provided with that chcice(Consequence). This Antecenden: -Behavior -Consequence, or 
A-B-C paradigm may be used to teach 

1. Highly reinforcing acrivities/objects which may be readily provided should be 

identified. 

2. Mildly aversive acrivities/objects which may be readily provided should be identified. 

3. NAME should be presented with two choices, one highly reinforcing choice and one 

mildly aversive choice. . 

4. The symbol set should be one with which NAME has demonstrated proficiency. 

5. Choices should be clearly presented and spaced so as to facilitate clear arid easily 
interpreted responses. 

6. Choices should be clearly labeled by the communication partner (i.e.. verbally state 

"NAME do you want or do you want ,?" while pointing to the 

appropriate choice in order to provide visual cues). 

7. NAME should then be expected to point to a choice in response to the prompt "What 
do you want?". 

8. When appropriate, NAME should be prompted to 'look where he is pointing in order 
to effectively pair the behavior with the consequence. 

9. The choice object/acrivity.should be immediately provided along with the verbal cue 

"You chose the Here's the ”. 

10. NAME should be required to interact with the choice for at least five seconds. 

11. The location of choices on the display should be varied in order to be sure that 
NAME is attending to the task and not the location. 

12. Following repeated trials over several days, if NAME clearly avoids the mildly aversive 
choice, it will be evident that he understands the task. 

It is important to differentiate this communication program, requiring the participant to 
actively make choices in order to encourage functional communication skills from a 
language program which requires the participant to passively identify labeled objects and 
pictures. • 
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Toys for Youngsters with Handicaps 

• Resources • 

Below is a listing of some excellent resources for toys and developmental tools for 
use by youngsters with handicapping conditions. It is highly recommended that 
selection and application be guided by a professional familiar with the child. 

• Ablenet 

Cerebral Palsy Center, Inc. 

360 Hoover Street N.E. 

Minneapolis, MN 55413 
(612) 331-5958 
Contact: Cheryl Waite 

• Adaptive Communication Systems, Inc. 

Box 12440 

Pittsburgh, PA 15231 
(412) 264-2288 

• Don Johnston Developmental Equipment 

P.O. Box 639 

1000 N. Rand Road- Bldg 1 15 
Wauconda, IL 60084 
(312) 526-2682 

• Prentke Romich Company 

1022 Heyl Road 
Wooster, OH 44691 
(800) 642-8255 

• Simplified Technology for the Severely Handicapped 

Controls for Battery operated tovs and computers. 

Linda J. Burkhart 

8503 Rhode Island Avenue 

College Park, MD 20740 s . 

(301) 345-9152 

• Steven Kanor, PhD., Inc. 

8 Herkimer Avenue 
Hewlett, NY 11557 
(516) 783-6483 

• TASH, Inc. 

70 Gibson Drive, Unit 12 
Markham, Ontario, Canada L3R 4C2 
(416) 475-2212 

• Therapeutic Toys, Inc. 

91 Newberry Road 
East Haddam, CT 06423 
(800) 638-0676 

Michael Skubel 2/89 
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THE FOUR TOOLS: A READINESS FOR SYMBOLIC COMMUNICATION 




O 

ERIC 



30 



4 



Transitioning Through Representational Hierarchy 
Page 3 



Envirotech and Ultra-Glo (Chemco) (available at craft supply stores) can be used to 

Mmm“nTc«ion) re A^the T SmaH rea ‘ ° bjeCtS SUCh 35 Cherri ° S (Mussel white, personal 

miniature th°s disnlav L COa ^ n§ process retains ^ contour of the mounted 

miniature, this display format is well-suited for use with visually-impaired Clients. 

3. Consider using "object-pictures" to effect a smooth transition from real objects to 
graphic representations of real objects. oojects to 



REAL OBJECTS 



V 



4 



OBJECT-PICTURES 



V 



PICTURES 



Example 1 
Drink 



Example 2 
Cereal 



Example 3 

Kiss 





r 



"0“ 



one serving site plastic lips 






EJ 


IE 


& 


£7 


0 


0 


£7 


o 


Q 



cross-section of real object is mounted 
on a surface to create a three- 
dimensional picture. 



picture "figure" is raised in relation to 
the picture "background" on which it is 
mounted; height of the raised picture is 
gradually decreased over time 



fiat picture is achieved 



size of picture is gradually decreased in 
keeping with the individual’s motor/ 
targeting abilities 
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ASSESSING THE ABILITY OF VERY YOUNG AND LOW- FUNCTIONING STUDENTS TO USE 
SWITCHES AS A TOOL FOR CONTROLLING COMMUNICATION DEVICES 



Gail M. Van Tatenhove 



Purpose 

To assess the student's development of means -end and causality concepts 
relative to the use of a switch-controlled communication device. 



Underlying Assumptions 

Theoretical Premises * 

„ 1. Functional use of an electronic communication aid involves more than 

the mechanical operation of the device. 

2. Many teachers and clinicians expose students to the concepc that 

switches can be used as tools by connecting the switch to a battery- 
operated toy. While this procedure may be a valid means to teach 
switch-end device association, it does not promote the communicative 
use of a switch-controlled augmentative communication aid. Students 
who recognize the switch as a means to control a toy may or may not 
recognize that a switch can control a communication device which, in 
turn, can express a message that controls the behavior of a 
communication partner (see Appendix A for a hierarchy of tool use 
relative to communication). When assessing a student who may require 
an electronic communication device in the future, clinicians should 
consider what tool-use skills need to be developed. For example, if 
the student needs to use an interface (e.g., headpointer) , does he or 
she understand its use as a tool? Does the student recognize the 
recommended augmentative communication aid as a tool to communicate 
ideas, express feelings, etc.? 

Client Characteristics 

1. Age Range: 12 months to 2 years (cognitively). 

2. Developmental Level: Infant, preschool. 

3. Disability Type: Orthopedically impaired, with or without learning 

impairment (MR/LD) ; mildly/moderately/severely intellectually 
impaired; autistic. 

Setting: Evaluation center. 
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Augmentative Aids, Symbols, and Techniques 

Low- tech aids: Battery operated toys with a variety of single-action 

switches. 

High-tech aids: Simple scanning device (e.g., Zygo 16, Versascan,^ Light 

Talker with single-icon messages^). 

Techniques: On/off; single row/column scanning. 



Description of Strategy 
A. Selecting an Appropriate Switch 



1. Interview the student's occupational therapist, parent, and teacher to 
determine previous experience using a switch to control equipment or 
appliances, such as a remote control for the television. Observe, if 
possible, the student's use of a switch. 

2. If the student does not have previous switch experience, consult with 
the occupational or physical therapist to determine the student's best 
access mode (i.e., reliable body part and kind of movement to use). 

3. Select a switch that has a low cognitive demand using the following 
hierarchy as a guide: 

Easiest to Hardest 



Student must see the 
connection between the 
switch and the result 
(e.g., tread switch 
activated by hand with 
the cord visible) . 



Student requires some 
feedback from the 
switch but does not 
need to "see the 
connection” (e.g., 
tread switch with 
auditory feedback 
activated by body part 
he or she cannot see) . 



Student can use a 
switch with limited 
feedback, that has no 
obvious connection to 
a device, and is 
activated by a body 
part not seen (e.g., 
light pointer) . 



B. Setting up an Appropriate Switch and End Device 

1. Select an on/off device (preferably a simple toy) that will not be 

overly stimulating to the student (e.g., a tape player that is on when 
the student activates the switch and is off when the student releases 
the switch) . 



^Available from Zygo Industries, P.O. Box 1008, Portland, OR 97207. 
^Available from Prentke Romich Co., 1022 Heyl Road, Wooster, OH 44691. 
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2. Make whatever compromise is necessary between the access mode and 
switch/device selected to provide the best set-up with the least 
cognitive demand. For example, a student with severe cognitive as 
well as physical limitations may need a switch by the head. However, 
if the student cannot see when the switch is activated, the cognitive 
demand of the task may be too high. Therefore, the OT or PT may need 
to assist the student in activating the switch to turn on or off the 
device. Caution: The OT/PT must wait and feel the student make a 
volitional movement before helping the student to activate the switch. 

3. Once the switch and end device is set up, allow the student to play 
and experiment with it. Usually two to five minutes are adequate. 

C. Assessing Potential Switch Use for a Communication Aid 

After students have experience with the switch and end device, analyze 
responses according to the following scale: 

Response //l - student focuses attention (measured through eye gaze) on his 
or her action on the switch, appearing continually surprised at the 
result on the end device. 

Response #2 - student alternates gaze between the action on the switch and 
the resulting effect on the toy, appearing to develop a beginning 
association between the two. 

Response #3 - student maintains attention on the end device, disregarding 
his or her action on the switch. 

If the student consistently repeats response #1 (maintains gaze on switch 
on 75% of trials), continue switch-to- toy and switch- to -recreational computer 
experiences. Reassess after continued experiences with a switch for 
recreational purposes to determine when th.e student can use a switch to control 
a communication device. 

If the student demonstrates response #2 or #3 on 75% of the trials, 
evaluate the student's ability to use a communication device. 

D. Facilitating Switch Use Understanding 

Switch-end device use can be facilitated, if necessary. For example, if a 
student simply activates a switch to control an on/off toy, significant 
goal-directed behaviors are not involved (i.e., action/no action is all the toy 
can do) . • A different toy may permit the child to understand a more advanced 
concept. For example, a toy bear that climbs can be made to climb to a 
designated location. In this case, the student must understand the goal and 
activate the switch until the goal is achieved. 
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By changing variables involved in switch- end device use, switch use can be 
enhanced. For example, change the 

1. end device; maintain same mode of operation (e.g., simple on/off). 

2. type of switch; maintain the same mode of operation (e.g., momentary 
single action^) . 

3. access site of the switch (i.e., move the switch to a second, 
approprate body part) for a more timely, reliable access, if 
necessary. 

4. type of switch and the mode of operation (e.g., dual action or 
latching switch^) . 

5. end device and the mode of operation (e.g., a goal-directed toy that 
requires repeated activation of the switch) . 

The student's responses to changes in these variables permit the 
facilitator to determine at what level to begin switch activation training. 

For example, if the student continues to exhibit response //I after all 
variables are changed systematically, the facilitator will provide additional 
experience at that level. The student would not be ready to use a switch- 
controlled communication system. Continued recreational use of a switch with a 
variety of end devices would be required. 

When the student demonstrates responses #2 and //3 when presented with 
variables described, the student's ability to use a specific communication 
device in a functional activity may be evaluated. 

E. Evaluating Functional Use of a Communication Device 

1. Use a simple step-scanning device with a momentary action switch. Use 
the switch access mode th^t has proven best for the student. Place a 
stimulus item, such as a small object or photograph, in a target area. 
Functional and fun objects, such as wind-up toys, balloons, and 
bubbles, work well. 

2. Ask the student to advance the scanning light to the designated target 
area. When the light reaches the target, give the student the object 
as a meaningful reward. 

3. Place a second, less motivating stimulus item in a second target area 
(keeping the first item in the original target area) . Ask the student 



3 

A momentary single action switch is similar to a doorbell. It has an on-off 
action. 

^A dual action or latching switch turns on when activated and remains on when 
released. 
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to choose between the two objects by moving the light to the desired 
object. Give the student the object selected. 

4. Record the amount of prompting and guidance necessary for the student 
to use the device in a functional, communicative manner, including: 

(a) the cues needed for the student to understand that he or she needs 
to advance the light to the designated target; (b) whether the 
student directs eye gaze to the examiner or vocalizes upon reaching 

..the desired target area; or (c) whether the student reverts and no 
longer understands the relationship between the switch and the end 
device . 

5. Using clinical judgment, provide additional trials, add additional 
stimulus items, vary the location of stimulus items, provide 
additional practice time, etc. 

6. If the student does not demons trate ‘ functional , interactive use of the 
communication device, continue to use switch-controlled, recreational 
devices. Consider use of a communication device only in a controlled 
treatment setting using activities that will promote communicative use 
and understanding of the device. 

7. Use a switch-controlled communication device in a variety of settings 
if the student demonstrates functional interactive use of the device. 
Provide interaction guidelines to professionals, parents, siblings, 
and peers working with the student. 



Evaluation of Effectiveness 

The procedure was validated clinically. Approximately 50 students were 
screened over a six-month period. An analysis of the responses of five 
students at each of the three response levels was made to compare the type of 
responses observed and each student's functional use of a communication device. 
Then, one student at each response level was selected and followed for 14 
months. Results were consistent in that each student progressed through 
understanding the use of switches as outlined. 
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Appendix A 

HIERARCHY OF TOOL USE RELATIVE TO COMMUNICATION 



ACTS INDEPENDENTLY: noncommunicative 

o uses body - grabs, goes, gets, looks at 



EXTENDS USE OF BODY TO GET ANOTHER TO ACT: communicative 

o points to object and looks at person 
o looks at object, then to person, and back to object 
o takes hand and shows 



USES TOOL TO GET ANOTHER TO ACT: communicative 

o looks or points at object/picture on device and looks at 
partner 



USES TOOL ON TOOL TO GET ANOTHER TO ACT: communicative 

o uses headpointer, T-bar, or light source on head to point to 
symbols on the device and, thereby, communicate ideas 
intentionally < • 
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SUGGESTIONS FOR STIMULATING VOCAL IMITATION 




Vocal imitation skills lay the groundwork for later conversational turn-taking. When 
attempting to establish vocal imitation skills, care should be taken to follow the normal 
developmental sequence. The following are suggestions for encourgaging the 
development of. vocal imitation skills. 

1. Make a conscious effort to playfully imitate sound(s) that your Child has just 

produced. Be sure to make it a game. At this time, your Child may have several 
sounds/sound sequences within his/her repertoire that you can imitate. Anv sound 
that your Child produces can become the focus for vocal turn-taking. 



Examples: 

Child: Baba* 

Caregiver Baba 

Chil± Baba 

Caregiver Baba, etc. 




2. With time, you should be able to start the turn-taking sequence using sounds/ 
. sound sequences that your child frequently produces (i.e., familiar sounds). 





*The examples used in this program are only examples and may not be sounds 
within your Child’s repertoire. 
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Stimulating Vocal Imitation 
Page 2 




Examples: 



3. 



Caregiver 

Child: 

Caregiver 



Baba (knowing this is a sound sequence that your Child 
frequently produces) 

Baba 

Baba, etc. 





Later, when the Child is able to reliably perform suggestion 2 for several 
sounds/sound sequences in his/her repertoire, try switching off between different 
sounds in his/her repertoire. 



Examples: 




Caregiver 


Eeee'(knowing this is a sound sequence that your Child 
frequently produces) 


Child: 


Eeee 


Caregiver 


Eeee 


Child: 


Eeee 


Caregiver 


Baba (knowing this is a sound sequence he/she frequently 
produces) 


Child: 


Baba 


Caregiver 


Baba, etc. 



Much later in intervention when the Child is able to reliably perform suggestion 
3, try switching off to new sound sequences (i.e., sounds that are not currently in 
his/her repertoire, sounds you’ve never heard him/her produce before). 

Examples: 



Caregiver 

Chili 

Caregiver 

Chili 

Caregiver 

Chili 



Eeee (a sound sequence that your Child typically produces) 

Eeee 

Eeee 

Eeee 

Boo-boo-boo (a sound sequence that your Child does not 
typically produce) 

Ba-ba-boo 





NOTE: Initially the Child’s imitations may not be entirely accurate. With 

time, however, he/she will acquire greater skill in matching his/her 
vocalization with yours. 
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Stimulating Vocal Imitation 
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5. At this point, the Child should already be attempting to imitate some of your 
words. 



Handout developed by: 

© Carol Goossens’, Ph.D., CCC-SP 
Sharon Crain, M.S., CCC-SP 

Sparks Center for Developmental and Learning Disorders 
University of Alabama at Birmingham 
P. O. Box 313, University Station 
Birmingham, Alabama 35294 

# 
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Augmentative 
& Communication 

News 



INSIDE 

THIS ISSUE . . . 



For Consumers 




Clinical News 

: i : 



How humans develop skills: 
Information processing and 
visual scanning 




Considerations for training 
visual scanning techniques 



Equipment 




Governmental 








"You say regular; they say. automatic" 
Let’s call the whole thing scanning! 



Updating the Tech Bill: 
The ball is in Washington’s court 



University and Research 






UPFRONT 



This issue focuses on Visual 
Scanning. I was fortunate to speak 
with many individuals with exper- . 
tise, information and exciting ideas 
about the topic (see Resources). 
Some interviews were conducted in 
New Orleans at the end of June 
while attending the first meeting of 
the U.S. Society of Augmentative 
and Alternative Communication 
(USSAAC) held in conjunction 
with RESNA. Very stimulating, fun, 
and HOT in New Orleans! 




For Consumers attempts to re- 
late what is known about how able- 
bodied humans "learn complex 
motor skills" to the experiences of 
those attempting to develop visual 
scanning techniques. 




Sweden: Current research 
& development activities. 



Clinical News focuses on issues 
and practices relevant to teaching 
visual scanning techniques. In 
Equipment, you'll find a brief 
description of available scanning 
communication aids and computer 
software that can facilitate the 
development of various types of 
scanning skills. 

In University/Research ongoing 
research activities in Sweden are 
highlighted. For those planning to 
attend the 4th International Aug- 
mentative Alternative Communica- 
tion Conference in Stockholm next 
August, the article may help you 
plan your August, 1990 "working" 
holiday in Sweden. 

The Governmental section pro- 
vides a brief update on the status of 
the Tech Bill in the U.S. (cont.pg. 2) 
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jx For Consumers 

v# ; Visual Scanning: 

;j t j What's it all about? 



Individuals who use visual scan- 
ning techniques typically have 
severe motor impairments that 
preclude and greatly restrict their 
direct access to communication 
devices and computers. Included 
are those with cerebral palsy, 
degenerating neurologic conditions, 
traumatic brain injury, and others. 

Although visual scanning does in- 
deed provide access, it is notorious- 
ly slow. Therefore, some consider it 
a choice of last resort. However, 
visual scanning can be and is a 
feasible option - worthy of con- 
sideration. 

* Visual scanning may be less fatiguing 
motorically than direct selection techni- 
ques. For some, it is faster. 

* Visual scanning may be accomplished in 
positions where direct selection access 
to devices is impossible, e.g., lying down. 

* Many software programs and communi- 
cation aids have visual scanning options. 

* It’s not an "either/or" proposition. All in- 
dividuals should use multiple techniques 
to participate in communication, educa- 
tion, leisure, and vocational activities 
and to control their environment. 

Visual scanning is a skill, and 
skills are learned. 

Skill Development 

The development of motor skills 
is a complex process. Most of us 
haven't given much thought to 
what's entailed. Try to recall learn- 
ing (or trying to leam ) to ski, drive a 
car or wheelchair, type 9 use a light 
pointer, play racquetbalU dance , or 
operate a communication aid to con- 
verse with a friend! Because adults 
avoid learning new motor tasks, we 
tend to forget how complicated and 
difficult it can be. 2 

Models of complex motor learn- 
ing can help professionals and con- 
sumers think about tasks like visual 
scanning. Robb 3 (cont.pg.2) 
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of visual scanning skills is con- 
sidered below: 



Augmentative 
Communication 
News 

Upfront (from page 1) 

Lots of Hotline calls this M 
month: service delivery & public 1 
policy concerns, questions about ; 
aphasia software, requests for : 
ideas re: clients. Whatever ques- 
tions you may have, just call (408) 
649-3050. I’ll try to help. Please ; 

keep in mind, we are on Pacific 
Standard Time . Thanks! 

We hope you are enjoying the 
summer (or winter for subscribers 
down under.) We're planning a trip \ 
to N ew Zealand & Australia in J 

September. Remember, don’t take ! 
those electronic communication . . \ 
aids in the water. . . swimming! 4i| 

~~ a 

Visual Scanning (from page 1) 

characterizes skill development as a 
process, which occurs in 3 phases: 

Phase I - Plan formation* Initially, 
individuals must learn what is to be 
done and why . Phase I can take a 
few minutes, hours, days, or even 
months. This varies with the com- 
plexity of the task (e.g., type of scan- 
ning) and with the learner’s 
capacities and limitations. 

Phase IT - Practice: The key in 
Phase II is for learners to engage in 
meaningful, appropriate practice 
with feedback. The amount of prac- 
tice needed varies with the com- 
plexity of the task and the past ex- 
periences and capabilities of the in- 
dividual. The literature suggests 
able-bodied persons of different 
ages approach practice differently. 
Children are curious and easily 
challenged; they practice to "see 
what they can do." Adults want 
results quickly and hope to achieve 
without putting forth much effort. 

Feedback is the single most impor- 
tant factor. Simply doing something 
over and over again does not im- 
prove performance. We sign our 
names again and again , but I'll bet 
your signature hasn't gotten any 
more legible than mine overtime ! 



Feedback can play 3 roles: 
motivate , reinforce , and/or regulate 
behavior. Errors occurring during 
practice are necessary for learning. 
However, without meaningful feed- 
back (i.e., what caused the error 
and how to correct it), the learner 
is probably not going to progress. 
Simply giving positive reinforce- 
ment (e.g., saying "good job," or dis- 
playing "blinking blobs") may keep 
learners motivated and working at a 
task, but will not help them develop 
and refine skills! " Teachers who 
develop a discriminatory eye for 
detecting errors in sequential and 
temporal patterning are the best 
source of feedback."* 

Phase III - A utomatic_execution: 
The last phase begins when the 
learner performs the total move- 
ment pattern with fairly consistent 
results. Stress and anxiety are 
reduced because tasks have be- 
come easier. Constant monitoring 
continues, but is relegated to sub- 
liminal levels. During this phase 
complex skills become automatic, 
and individuals can concentrate on 
other factors (e.g., communication, 
the content of a term paper, im- 
plementing rate enhancement 
strategies, etc.) 

Research suggests that making 
major changes during Phase III, 

(e.g., changing the order of sub- 
routines or introducing a different 
subroutine) is very difficult for able- 
bodied individuals to accom- 
modate. This should make us 
cautious about presuming some 
sort of hierarchy to teaching visual 
scanning skills, (i.e., first introduce 
step scanning, then linear, then 
row/column, group/item, directed 
scanning.) To date, however, no 
evidence exists that training one 
type of scanning facilitates or inter- 
feres with learning others. 

Capacities/ Limitations of Man 

Information about human 
processing capabilities and limita- 
tions that may affect the acquisition 



1. The sensory capacities of 
learners (i.e., depth perception, 
peripheral vision, acuity) are 
generally nol improved through 
practice. Comment: Many in - 
dividuals with severe motor 
problems have visual acuity, percep- 
tion, field cuts, and/or fixation dif- 
ficulties. This will profoundly affect 
their ability to use visual scanning. 

We often must look for ways to com- 
pensate for, rather than remediate 
these problems . 

2. To detect signals, humans must 
distinguish the figure from the back- 
ground. Loud noises, novel stimuli, 
and verbal praise can heighten the 
ability of humans to detect signals. 
Comment: The stimulus (e.g., cur- 
sor, LED, pointer, target) should be 
the most obvious thing around. 

Other aspects should be minimized 
during training. Learners must be 
able to maintain their gaze on targets 
while they activate switches. 

3. Selective attention is important 
to the development of complex 
skills. Humans can (and do) filter 
out irrelevant information. Com- 
tnenti P ersons with central nervous 
system problems and young children 
often have difficulty filtering out 

" irrelevant " stimuli (e.g., " attention 
problems ," n distractibility, n "hyperac- 
tivity.") Setting up contexts that allow 
learning to occur is critical. 

4. Perception, i.e., interpretation of 

sensory information, is influenced 
by instructions and an individual’s 
experience with a task. Also, feed- 
back affects perception (see Phase 
II). Comment : Individuals who are 
learning new skills need all 3 types of 
feedback. ' ' 

5. Vigilance tasks depend on a 
person’s monitoring capacities. 
Monitoring capacities are depend- 
ent on how much, how fast, how 
predictable stimuli sources are, as 
well as on the memory capacity of 
individuals. If events occur slowly, 
learners can more easily recognize 
and process sensory input. Tasks re- 
quiring vigilance are acquired more 
easily with frequent rest periods. 
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Comment: Instruction s, demonstra- 
tions , and practice sessions must be 
controlled Long practice sessions 
may not be productive . Rate enhan- 
cement techniques require vigilance . 

6. Reaction time (i.e., time taken to 
initiate or begin a movement) is af- 
fected by learning, anticipation, and 
the amount of information people 
must process, as well as their motor 
capabilities. The more choices 
someone has, the longer the reac- 
tion time. Comment: Visual scan- 
ning array’s typically contain an enor- 
mous amount of information . If 
someone must search for targets , 
reaction time will increase . Dynamic 
arrays and encoding techniques also 
increase the amount of information 
processing required and will affect 
reaction times . 

7. Movement time (time between 
initiation and completion of move- 
ment) is affected by the need for ac- 
curacy. Able-bodied humans can 
and do trade off speed and ac- 
curacy. Comment i Activating a 
switch involves reaction and move - 
merit time. Thus , motor planning 
plays a major role in the speed and 
accuracy of visual scanning skill 
development 

Performance of Able-bodied 
Persons on Scanning Tasks 

Direct selection vs. scanning . 
Ratcliffs compared the accuracy of 
able-bodied children using direct 
selection and scanning techniques 
on the Light Talker (1st through 
5th graders) on a direction-follow- 
ing task. Results indicated a sig- 
nificantly greater number of errors 
for all those using scanning . 

Visual vs. Auditory Scanning . Fried- 
Okens compared the performance 
of 90 adults using 3 scanning ap- 
proaches on a sentence recognition 
task: visual only, auditory only, and 
visual/auditory. The task was 
designed to simulate typical condi- 
tions for communication device 
users. Results showed response ac- 
curacy and reaction times were sig- 
nificantly better for the visual scan- 
ning only c ondition . Auditory scan- 
ning was the poorest and combined 
scanning fell between the two. A 
Makes sense! 



pnEi Clinical News 

!-:! Visual Scanning: 

Training Approaches 

V isual scanning is the sequen- 
tial presentation of choices (letters, 
words, or pictures). As options are 
presented by a person or machine, 
the user signals (i.e. looks up, vocal- 
izes, pushes a switch) just at the 
"moment” the desired item (or 
group, which contains the item) is 
highlighted. Master teachers, 
clinicians, researchers with whom I 
spoke, agree visual scanning techni- 
ques involve multiple skill com- 
ponents: motor, motor planning, 
visual, perceptual, kinesthetic, cog- 
nitive, and linguistic. Pripr to intro- 
ducing visual scanning techniques, 
certain assumptions are made: 

1. Individuals are positioned in way s 

that allo w, and facilitate, function. 

Seating specialists often focus on 
adaptive seating for wheelchairs; 
however, communication occurs 
everywhere so other positions are 
considered. Steps toward position- 
ing the body to augment function 
are summarized in Table I. 



Table I. Positioning for Function 

STTPS STRATFCiTRSTO CONSIDFR 



l* 1 Stabilize pelvis 
; (normalize tone, & 
reflexes) 

I 2 nd Stabilize trunk 

ri 

•;! 

3 Stabilize arms 

;t 

[ 4^ Stabilize lower 
il extremities, feet 



Seat belt; Type of cushion; 
Contour and angle of insert 
(e.g., allowing gravity to assist) 
Angle at hips (generally 90°), 
Midline orientation: Shoulder 
harnessing; Strapping; Lap 
tray, Side supports 
Arm restraints; Protracter pads 
Lap tray adaptations, Dowels 
9u at ankle and knees 
Orientation of insert 
Ankle straps; shoe cups; calf 
straps; toe loops; foot rests 



! 

j 



j 



2- An appropriate inte rface is estab- 
lished. Professionals with exper- 
ience and expertise in assessing the 
human/machine interface (e.g., 
rehabilitation engineers, O.T.s, 
some educators and speech-lan- 
guage pathologists) make these 
decisions in consultation with the in- 
dividual and family. Anatomic sites 
under the control of the individual 
are identified, and movement pat- 
terns described. Abnormal reflexes 
often are a major consideration. 
Finally, interface/switch options are 
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assessed and selected. Assessment 
software can be helpful. 7 Table II 
lists questions to get answers to. 



Table II. Questions to consider 

Docs the person activate the switch? How • 
long does it take? Can the person hit 
without prehits or multiple activations. j 
Can he activate it at a particular time? • 
Does he stay oh the switch? How long? 

Can he stay off the switch? Can he release 
. the switch on verbal command? on a 
visual target? Can he coordinate the 
timing of activation and release with the 
; visual components of task, i.e., tracking, 

; fixation, etc.? Can he see the display and ; 
; activate the switch? Does he fatigue? j 



3. ThepositiorLPndstabilitv of the 
swildmnd the display is optimized. 
This is an often overlooked, yet 
critical step. Visual, perceptual and 
motoric factors are pertinent. 
Decisions are made re: how far 
away and at what angle the 
switch(es) and visual display is 
placed and mounted. Ways to 
mount and stabilize switches are 
available from commercial 
manufacturers/distributors. Get 
some help! Goossens* and Crain in 
their wonderful new book describe 
success using chlorinated polyvinyl 
chloride (CPVC) pipe mounts. 8 

4. Major alterations in the position 
of individuals, switches . and the 
device! display are completed prior to 
implementing high technolog y Scan- 
ning training often begins with no 
tech or low tech approaches while 
positioning/seating and interface 
decisions are being made. 

5. If individuals do not understand 
the task^of scanning, yet need a 
means_of interacting with some inde- 
pendence . L a conceptual bridze can 
he built through instruction. To do 
so, increase a person’s familiarity 
with the task demands by maximiz- 
ing important features, presenting 
important sequences and relations 
in a coherent whole, & minimizing 
information load by stripping the 
task of all but essential elements. 9 

The level at which individuals 
can accomplish different scanning 
techniques is unknown. Some say 2 
year olds can use switches to make 
simple choices. Normal 3 year olds 
can use the Versascan (cont.pg. 4) 
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with remote lamps and the Zygo 
16. 9 Light’s 1 ^ research shows linear 
scanning requires two coordinated 
motor/cognitive operations 
(monitoring the relation of the cur- 
sor movement through a visual 
array of pictures and timin g activa- 
tion of the switch to make a selec- 
tion) and concludes it is probably a 
4 year level task that can be taught 
to those younger. 

• She analyzed the performance of 3 
children '(experts) who successfully used 
automatic linear scanning to access a 
computer (2 were able-bodied, 1 was 
nonspeakingand severely physically dis- 
abled.) Errors were similar, i.e., an- 
ticipating the cursor's movement and ac- 
tivating the switch prematurely. The 7 



able-bodied children visually tracked the 
cursor, while the physically disabled 
child focused on the target, a more ad- 



cursor. while the 
child focused on 
vanced strategy. 

One thing is for sure. Clinicians dis- 
agree. Some report 3 1/2 year olds 
learn row/column sc annin g. Most 
report success only for those above 
4 years old (developmentally). Few 
ventured a guess for directed and 
block scanning.. .maybe 7 year olds? 
Researchers! We need to know 
when to build bridges and how. 

Task Analysis 

In the past, professionals (77 n 
guilty) taught visual scanning techni- 
ques without first analyzing the task 
or thinking beyond its motor com- 
ponents. Steps to a task analysis are: 

• 1. Always try the task yourself. 

• 2. Ask others who have mastered it, how 
they do it? Observe what they do. 

• 3. Break the task down into the motor, 
visual, cognitive, linguistic, social re- 
quirements, Then, consider how com- 
ponents are coordinated. 

• 4. Develop a training strategy based on 
what you know about the equipment 
(i.e., swucta, software, hardware) and 
nave observed about a client. 

• 5. Implement your strategy, evaluate the 
effects, and make modifications. 

Training Guidelines 

The 3 Phases described in For 
Consumers provide the framework 
for this section. Assessment is treat- 
ed as an integral and ongoing part 
of the process of skill development 
so is not considered separately. 

I. Understanding Scanning 

Phase I emphasizes cognitive fac- 
tors, minimizing motor, linguistic, 
social, and visual components. 
Whereas young and mentally hand- 
icapped individuals may require 
months of training, older children 
and adults will catch on after a brief 
demonstration and description. 



• Demonstrqriops Showing the sequential 
ordering of subroutines and giving infor- 
mation about the purpose or objective 
of the task. 

• Descriptions Communicating about 
what learner is to learn (e.g.,Tm push- 
ing the switch I’m watching the target. 
Here comes the cursor. There, I got it!). 

Because visual scanning skills 
develop at a young age without tech- 
nology (e.g., watching people, locat- 
ing and looking at desired objects, 
making choices), we often begin 
with a "low/no tech" emphasis. For 
example: 

• Linear scanning of objects, pictures, 
people. "Let me know... do you want this, 
this, or this? Individual indicates "Yes" 
as you point to choice by smiling, vocaliz- 
ing, looking up, touching a buzzer, turn- 
ing toward a red pad, etc- 

• An Etran or Eye-Gaze Vest which 

f >rpvides communication and language 
raining also can be used to introduce 
visual scanning techniques (i.e., locating 
targets, making selections.) 

• Linear, row/column orblock/item scan- 
ning using symbol, word, or alphabet 
boards can easilvbe introduced as lis- 
tener assisted techniques. 

Few have difficulty under- 
standing what a switch does. If 
there is a problem, consider the fol- 
lowing: 

• VanTatenhove 11 describes 3 levels of 
responses from young and severely men- 
tally handicapped children: a) Child 
focuses on switch with limited, if anv, 
awareness of its relationship to anything, 
else* b) Child focuses both on switch ana 
the effect" with awareness of the 
relationship, c) Child sees switch as a 
way to make things happen and focuses 
on the goal (e.g., making screen change; 
advancing slides in projector, activating 
a loop tape to call friend). 

• When switches are a distraction, make 
the consequence of switch activation 
(e.g., visual display, fan) more obvious 
and the switch invisible. For example, 
use a computer and turn off the lights. 

When you plan to introduce a 
specific device or computer pro- 
gram that uses sc annin g, adults and 
older children will readily grasp the 
task. For younger children, 

• games, play activities, and familiar 
routines provide learning contexts. Com-- 
puter software and training aids can be 
very helpful (see Equipment). 12. 13.14 

If you can’t tell what someone 
understands, Cook 15 describes a 
protocol that can help: 

Observe motor behaviors in 
response to: a) non time-contingent 
switch activation (prompt free); b) 
hold/release activities; c) time con- 
tingent switch activation; d) 
hit/release activities; and e) choice 
making. 

II. Practicing Scanning 

Everyone needs lots of practice 
in Phase II. Adults, like children, re- 
quire carefully thought out interven- 



tion approaches to skill develop- 
ment. 

In Phase II, tasks with low cogni- 
tive/linguistic/visual loads are used 
to allow motor and motor planning 
skills to develop. Clinicians monitor 
fatigue and positioning of the in- 
dividual, switch, and display before, 
during, and after practice sessions. 
Even small changes can affect per- 
formance. 

• Teach clients to go to a "home base" be- 
tween switch activations. Make it a real 
resting spot.8 

• Help individuals locate and/or fixate on 
targets by pointing, highlighting with a 
light, blocking out other parts of the dis- 
play, and/or providing verbal, auditory 
and visual cues. 

• Give individuals time to experiment. 

• To deal with anticipation problems, in- 
struct person to focus on the target. 
Slowdown the scan. Blocking out parts 
of display completely, at first, then 
gradually making parts more and more 
transparent mayhelp. 

• Provide meaningful feedback. This re- 
quires watching what individuals do and 
analyzing their behaviors, especially 
their errors. For example, Lightto 
describes a seemingly. random pattern of 
switch activations, wnich, on closer 
analysis, revealed that switch activations 
occurred each time the facilitator spoke, 
with little regard to the cursor position 
on the screen. Further analysis of the 
problem showed the person could indi- 
cate with a "yes" response when a target 
picture was pointed to and could push a 
switch in response to a visual cue, but 
had difficulty combining these opera- 
tions. 

• Use a simple training device with voice 
output, if possible. Person can select 
clothes for a doll, trucks for race, in- 
gredients for a cake, or what lipstick to 
put on you (e.g., green, purple). 

Use peers to model switch use and play 
games. 

How we arrange the visual dis- 
play will help determine the success 
of training. Table III provides 
guidelines. 



Table III. Optimizing Scanning Displays 

1 Place targets at individual’s midline (note: may be to 
the nght or left of "true" midline) 

1 Ask individual and vision specialists hcrw far away anc i 
what size displays/symbols should be. What individual: 

does in order to see. Is color important ?i6 i 

1 Consider placement of most frequently used targets * 
(vocabulary) with regard to cognitive, motor and lan- i 
guage demands. How will individual sequence lan- i 
guage? ij 

1 Figure out which are easiest to most difficult posi- i 
tions to access on an array. Start by asking person to i 
get to the easiest cell on the arrav. rs’everbegin wiih i 
row 1, column 1. Be very systematic about how you 1 
progress, . 

Think of the future (e.g, will number of locations ac- i 
cessed increase or decrease?). ; 

Take into account the type of scanning and whether i 
or not encoding or other rate enhancement techni- j 
ques will be overlaid on the scanning task. 



In the beginning, speed and ac- 
curacy are of little importance. 
However, once an individual can ac- 
complish the task wi.h (cont. pg. 5) 
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Table V. Communication * 

programs ; 

Communication Enhancement Center 

Boston. MA 

575 Trains linear, row/column, * 
stepped, directed scanning. 4-49 ■ 

locations. 

Sc in & Speak 575 Adds memory 
caoaoilitv to Target. 

RehabiLittion Engineering Center, Palo 
AJto, CA 

Slcn fiy Step 3100 Choice making, 9 cells. 
ENABLE Syracuse, NY 

O.Tnh.ils. S150 Semantically 
organized pictorgraphic symbols. 

Construct phrases 

full of good clinical, as well as tech- 
nical ideas. The Talking scan op- 
tion is very powerful 

Table V lists some communica- 
tion programs available for the 
Apple II family. They provide in- 
dividuals with exposure to visual 



scanning techniques as a means to 
accomplish communication tasks. 

Communication Devices with 
Visual Scanning Options 

Manufacturers offer an increas- 
ing number of visual scanning 
devices. Single dimensional devices 
are typically considered "training or 
educational aids" because of 
vocabulary restrictions and por- 
tability factors. Multidimensional 
devices are commonly chosen to 
serve the conversational, computer 
access, writing needs of individuals 
who are unable to use more direct 
methods. Tables VI and VII take a 
snapshot of what is currently avail- 
able. 

^ * 

1 
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' - ••••'ll." Update from 



Washington 



States in the U.S. wishing to be 
granted monies alloted through 
P.L. 100-407 to implement an assis- 
tive technologies program for their 
citizens await the results of the com- 
petition. Awards will be announced 
sometime this summer, early fall 
Also a technical assistance contract 
will be made soon to an organiza- 
tion/agency to assist states in im- 
plementing their plan. Cohen, 
Rehabilitation Program Specialist, 
NIDRR cautions states not receiv- 
ing grants to "'get ready." Year 2 
competitions will be coming soon. 
Call Cohen at (202) 732-5066. 



Table VI. Single Dimensional Visual Scanning Aids 



PRODUCT 


COMPANY 


PRICE 


TYPE SWITCH SPEECH 


PRINT 


Ljock Steven Kanor, Inc. 

CommunlcatorsS Main St Hading. 

On-Hudson, NY 10706 


560-76 


Rotary 


1 


No 


No 


Dial Scan 


Don Johnston Devel. 
Equip. P.O. Box 639 
Wauconda, IL 60084 


5195 


Rotary 


u 


No 


No 


Vers&scan 


Prentke Romich Co 
1022 Hcyt Rd, 
Wooster. Ohio 44691 


5799 


Rotary 


U 


No 


No 


Sequential 

Scanner 


Steven Kanor, Inc. 


571-86 


Linear/step 


1 


No 


No 


Scan Wolf 


AdamLab. 33500 Van 
Bom Rd. Wayne MI 
48184 


5375 


Lincar/step 


1 


Yes 


No 


Zvpo 16C 

t 

i 


Zygo Industries, Inc. 
P.O, Box 1008, Portland 
OR 97207-1008 


51145 


Linear 


1 


Yes 

w/ Parrot 


No 


. Steeper 

1 

! 


Zygo Industries, Inc. 


51445 


Omni 

directional 


1-2 


Yes 


No 



FEATURES 

Pointer Adjustable speeds; Several available 
with music and size option 

Pointer; Transparent, allowing face-to-face/ 
double— sided pictures; Gockwise/couter- 
clockwise; Variable speeds 
16 removable lamps, also remote lamps 
Translucent overlays, colored lamp covers 

2-4 compartments which light 

6 x 6 (36 selection areas); 30 levels 
Speech programmed at factory 

4x4 matrix 



Backlighted 4 x4 
30“ and 60 J 



matrix, Display can be angled 



j 

i 

j 

! 



i 

ii 



Table VII. Multi Dimensional 

Zygo Industries, Inc. 31950 Row/column 1 

31250/ Linear 1,2 
52500 Row/column 



Zygo Industries, Inc., 



S yitchfonrt Zygo Industries, Inc. 31495 



Zygo Industries, Inc. 31495 



Mb cm* 

(scannme) 

Kfln vv RIXhK Zygo Industries, Inc. 34,200 

. i L r * 1 5f' 3n (keyboard emulator only) 32250 

i-Xght TnIKrc Prentke Romich Co $4290 



lalk-O 



WSKF TT 

FZ C om 
Equalizer 

Special 

Friend 

(scanning) 



Tashlnc., 52,300 + 

70 Gibson Drive, # 12 access 
Markham,Ont.L3R4C2 option 
ACS $5,695 

354 Hookstown G r. Rd $3995 
Qinton, PA 15206 
Words + , Inc. 33395 

distributed by ACS above 
Words + , Inc 33395 

Words + 33395 

Shea Products, Inc. 53250 
1721 W. Hamilton Rd. 

Rochester Hills, MI 48309 



o 

ERIC 



Linear 1-5 

Row/column 
Directed 
Linear 1-4 

Row column 
Row/column 1-5 
Directed 
Linear, 1-5 

Row/column., 
Directed 
Linear 1-5 

Row/column 
Directed 
Linear 1-5 

Directed 
Block/item 
Row/col um 1-5 

Directed 
Row/column 1 
Row/column 1-5 
Directed 
Row/coiumn 1 



Visual Scanning Aids 

No No Limited memory, 16 messages 

Optional Yes Literacy required 

24 col. Conversation note taking, writing, and calculation 
Predicts based on frequency of use 
No No 4 to 49 locations; overlays for several options 



Yes 



No 



Optional Yes 
20 col. 



60 seconds high quality digitized speech 
120 seconds standard speech 
Various message storage options; calculator; 



Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 



Interface to computers. Literacy required 
Optional Express/M inspeak firmware w/rate enhancemen 



No 

Yes 



8, J2, 128 locations 

Multiple access options purchased separately, 

120 seconds of recorded speech 

Rate enhancement techniques.; 15 memory levels 



Optional Prediction, abbrcv. expansion, Literacy. \ 

keyboard emulator. Datavue Spark « 

Optional Designed for visual problems. Datavue Spark 1 
Optional Datavue Spark. Literacy. Prediction learns i 

user's patterns. Calculator, drawing, games, music » 
No 26 programmable pages 
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some degree of accuracy, begin in- 
creasing speed, or the number of 
items in the array, oi decrease the 
size of items. Change one thing at a 
time so you can determine its effect! 

III. Using Scanning to Accomplish 
Tasks and Achieve Goals 

Relying on scanning to communi- 
cate, complete homework assign- 
ments, requires being able to scan 
fast, without thinking about it. Auto- 
maticity (seen in touch typists, 
drivers, musicians, athletes) is 
needed. Speed and accuracy are 
major factors during Phase III, 
which never ends. 

* Accuracy: Some fed an accuracy level of 
at ieast 80% is necessary for operational 
competency in scanning. How accurate 
should someone be before you increase 
speed? We don’t know! Keep in mind, 
however, that even able-bodied people 
are not 100% accurate using row/column 
scanning techniques. 

* Speed ; Cook suggests thinking about 
thrcsnolds rather than absolute values 
(i.e.. range between the fastest and 
slowest rate at which client can select 
messages. Kraat reminds us when we in- 
crease' rates, we should provide clients 
time to accommodate to the increase. 

* Fatigue factors change during a day. 

Give options to alter speed ot scan. 

Scanning is a means to an 
end.... not the END. Ultimately, 
scanning techniques provide inde- 
pendent access to tools and enable 
individuals to participate in ac- 
tivities, interact, and accomplish a 
range of tasks more independently. 
Be patient. . . it takes a long time to 
develop complex skills. A 

^3 



News 

Hyper ABLE DATA for | 
. Macintosh computers with 20 | 

- megabytes on a hard disk will be | 
available soon. It is very, very j 
; nice. Assistive device products \ 
; and companies are listed in an a 
, easily assessable format that j 
makes sense, and can be as- j 
; sessed quickly. Price of the CD- ] 
; Rom version is S50 per year. 1/4" ij 
Tapes are $122. Floppies (20 of j 
them) are $199. 6-month updates i 
i included. This Data Base com- j 
; bines Trace Resource Books in- j 
formation with Able Data. For j 
further information or to order ? 
call (608) 262-6966. ? 



=j Efjj I j pm (1-2 switches.) Regular 

Uv^Ul^llIvlIli or inverse. Scan proceeds rapidly to the 
— *4 Techniques, vicinity of the target. User activates switch. 

Devices, and Software ? ure ° r P roceeds more siowiy until selection 



Single dimensional scanning 
techniques can be listener assisted 
or technology driven. They include: 

Circular Linear scan in a clockwise 
arrangement. Requires use of all eye 
muscles, a difficult visual/motor task. 

Pointers (which arc always visible) may 
make the task cognitively easier that lights 
(which blink and disappear). 

Step/Flement: Linear scan. User activates a 
switch to advance cursor/light/pointer 
toward target. Time "is a switch." When user 
pauses, location is accepted as desired 
selection, or a second switch may be used. 

Regular/automatic: User activates switch to 
begin linear (vertical or horizontal) scan and 
again to select item. Cognitively more 
difficult than circular because light/cursor 
disappears at end of row or column and then 
reappears. * 

Inverse: Linear scan. User holds down 
switch until cursor gets to desired item. 

Then, releases the switch to select item. 

To enhance rate and provide ac- 
cess to more vocabulary, multi- 
dimensional approaches are used. 
These are more complicated; errors 
occur more frequently. High and 
light tech options are possible. 

Row/column: Most commonly used 
technique in communication devices. User 
activates switch, device scans down rows 
until user activates switch, then device scans 
across columns, until user activates switch to 
select desired item. 

Block/itcm: Best example is alphabet 
configurations on the Adaptive Firmware 
Card. Linear scan. When cursor arrives at 
block containing item, user activates switch. 
Then, cursor scans items until user activates 
switch to make choice. 

Directed Scanning is a combina- 
tion of direct selection and scan- 
ning* It generally involves multiple 
switches, i.e., technology. Those in- 
terviewed feel we don’t pay enough 
attention to directed scan options 
in augmentative communication. 

Directed scan- User has multiple switches 
(e.g., joystick) controlling the direction of 
the cursor. Items arc selected when switch is 
released (i.e., inverse scanning.) 

Note : Use of the mouse/track ball is a kind 
of Tree form directed scan. Proportional con- 
trol devices arc not addressed here. 

Rate enhancement features are 
required if scanning techniques are 
to meet communication needs, par- 
ticularly conversation. Some op- 
tions employed are: 



LctterAv ord prediction: Rased on frequency 
or recency of occurrence, machine guesses 
letters /words as user creates message. A 
dynamic technique (i.e., user must 
constantly monitor the display and make 
decisions.) Interferes with automatirity. To 
date, evidence is limited re: performance 
effects. Woltosz reports impressive rates on 
a copying task.i 

Encoding: User accesses vocabulary and 
linguistic structures through codes, e.g., 
Minspcak, abbreviation expansion, etc. 
Because codes often must be memorized, 
automaticity is affected. To what extent this 
affects scanning rates over time is unknown. 

Software 

Many programs are available to 
teach scanning techniques. 12 
Programs listed in Table IV pro- 
vide opportunities for individuals to 
practice various types of visual scan- 
ning techniques on the Apple II 
family of computers, and, at the 
same time, have fun! Remember to 
be aware of what you are teaching 
by doing a task analysis before you 
use a program. 



Table IV. Scanning Training Programs -j 

;i 

Don Johnston DeveL Equip. Wauconda, I L 'j 

(312)5245-2683 ‘I 

Interacting games S65 Linear and row/column 
scans and switch training. 2 persons. 

Learn to Scan S60. Horizontal, vertical scans. 8 
programs with varying difficulties. .! 

Motor Training (lames S35 14 games covering a j 
wide range of scanning techniques. i 

Make It an d Make It in Time S60 each. 

Cause/effect, timingjudgement in time sequences, 
scanning techniques include group item and linear . 
scans. Children and adults. ; 

Rabbit Scanner & Run Rabbit Run S29.95/S39.95 
Horizontal scanning; timing 1 

RJ. Cooper & Assoc,Dana Point CA. (714)240-1912 

Early and Advanced Switch Games $30. Switch 
training. 

Switch progressions $37 More switch training, 
timing. 

Compulerade Products, Cardiff, CA 92007 
(619) 942-3343 

: Catch the Cow S13. Scanning trainer, 

j Cmch-nnd-Match $29.95. Follow-on to other 
'! training program. Also, a 2 to 20 choice array. 

! UCLA Intervention Prog. LA-, CA (213) 825-4821 

j Wh rrr is Puff. 1 go looping, ric-Talk SIS 

j each. Provide switch and scanning opportunities 



Perhaps the most powerful tool 
we have is the Adaptive Firmware 
Card. With the Apple II gs com- 
puters, its flexibility provides access 
to a wide range of software. The 
technical manual is (cont. page 6) 
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— University 

j ,-i &Researcn 

; Research in Sverige 

This issue highlights research 
activities in Sverige (Sweden) . . . 
the beautiful Nordic land where as- 
sistive devices are FREE!* Sweden 
will host the Fourth International 
Conference on Augmentative and 
Alternative Communication 
(ISAAC) August 12-18, 1990. Fig- 
ure 1 illustrates the location of re- 
search projects** mentioned 
below. Note: you may find this a use- 
ful guide if you travel to Sweden . 

In BROMMA, the Swedish In- 
stitute for the Handicapped, (P.O. 
Box 303; Bromma, Sweden, 5-757 
26) promotes and coordinates re- 
search and development activities 
in assistive technology areas. The in- 
stitute also evaluates and tests 
products, disseminates information, 
conducts research, and builds the 
knowledge base in Sweden. 

Projects include: Communica- 
tion support for persons with speech 
handicaps (E. Oisson) - interven- 
tions for those with aphasia, 
dysarthria, laryngectomy, etc. 

Come Ag ain (M. Lundman & M. 
Magnusson) - use of computer con- 
ferencing systems by communica- 
tion-impaired adults. 

Inventory of communication aids 
focthe mentally retarded nnd Com- 
munication for a pro f oundly mental- 
ly retarded adult (J. Brodin & L. 
Larsson) - use of technical aids by 
individuals with severe retardation. 

Evaluation of computer-based 
aids and software (I. Friman) - 
recommends products for approval 
and provides information. 

"Note: There is an assistive device center 
network that distributes approved assistive 
dcviccs/aids: 90 centers for nearing impair- 
ments; 30 centers for visual impairments; 35 
ccntcre for orthopedic problems; and 35 for 
mobilirv/communication impairments. Five- 
seven Regional Resource Centers are being 
established to work in close affiliation with 
existing centers. They will provide direct ser- 
vices to multihandicapped clients, and dis- 
seminate information, education, and techni- 
cal assistance to professionals. 

••Information used is from Re gistration of 
Nordic Projects relatin g to disanility ( 1988): 
Nord]cCom mittec on D is abili t y. 
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Figure 1. Sweden - 

In STOCKHOLM, The Royal In- 
stitute of Technology has always 
focused on technology applications 
for individuals with disabilities. 

AAC related projects within The 
Dept of Speech Communication 
and Music Acoustics (Box 70014 , 
Stockholm , Sweden S-10044 ), are: 
Speech synthesis for disabled per- 
sons (R. Carlson and B. Grans- 
trom) - Text-to-speech and Biiss-to- 
speech systems in multiple lan- 
guages. A current focus is develop- 
ment of different speech styles and 
speaker types (i.e., child) and addi- 
tional languages. 

Development of sp eech output for 
communicatioruind education and 
The use of computer and synthetic, 
speech. in teaching read ing and writ- 
ing (K. Galyas) - Applications of 
Multi-Talk and development of 
software to assist in teaching read- 
ing and writing, and in training 
those with aphasia, etc. 

Word prediction for mobility 
andfor language impaired (S. Hun- 
nicutt) - Software to assist in- 
dividuals unable to read/write to 
construct written messages. Users 
provide limited information (e.g., 

1st letter of words, no. of syllables.) 
A smart, natural language process- 
ing program in the computer "gues- 
ses" the word from a list of 10,000. 

Also in Stockholm is Stiftelsen 
ALA (Box 5410 Stockholm , Sweden , 
5-774 84). Projects which inves- 
tigate behaviors affecting and inter- 
ventions aimed at improving inter- 



action include: Investigating the 
relationship among motor abilities . 
qognitive development and com- 
municative ability in severely multip- 
ly handicapped (M. Granlund & C. 
Oisson) and Sodallnteraction in. 
special classes with mentally 
retarded (K. Gooransson)* 

Additional projects are aimed at 
developing teaching materials and 
methods, for specific populations. 
These are coordinated by the 
Swedish National Institute of 
Teaching Materials (Box 27052 
Stockholm , Sweden, S-102 57), and 
are carried out throughout Sweden 

• Deaf-blind children and adults. L* 

Forsfalt. (KRISTI AN STAD). 

• Multiply handicapped students using 

technical aids. G. Fcrsson. (UMEA). 

• Video & interactive video for mentally 

retarded. M. Liden (UMEA). 

• Computers in education of handicapped 

pupils. U. Garthclson & H. Hammariun 

(STOCKHOLM/BROMMA). 

In LUND, the Center for 
Rehabilitation Engineering (CER- 
TEC), Lund Institute of Science 
and Technology (Box 118, Lund, 
Sweden, S-221 00) has projects re- 
lated to AAC (e.g., A h ygie n ic suc k- , 
and-blow switch (I. Jonsson & L. 
Holmberg). They recently created a 
multidisciplinary tr ainin g program 
at the Institute. Goals are to help 
professionals specialize in the assis- 
tive technologies area and develop 
related research skills. 

In GOTEBORG, the University 
of Goteborg carries out research in 
the Department of Education (Box 
1010, Molndal, Sweden, 5-457 26, ) 
and the Department of Handicap ' 
Research (Brunnsgatan 2, Goteborg, 
Sweden, S-413 12). An example is: 
Communication in young children 
with a physical handicap andspeech 
impairment (E. Bjorch-Akesson). A 
longitudinal interaction study of 7 
preschool children and parents. 

Also, at the Statens Institute for 
Laromedel (RPH-RH, Box 21071, 
Goteborg, Sweden, S-400 71 ) the 
Software development c & methods (or 
children with severe motor problems 
project (E. Landin, E. Oisson) is ^ 
ongoing. T 

Thanks to Lars Augustsson. Gunnar Fagcr- 
berg, Sheri Hunnicutt. Elisabct Oisson tor 
sharing information about Sweden at the 
recent kESN A conference in New Orleans. 
Many thanks also to Karoly Galyas. 
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HINTS 

! 1. The portable DECtalk is now 
j available. It weighs 9 pounds, is 
i battery operated, and is com- 
patible with portable communica- 
tion aids and personal computers 
with an RS232C port. Price, 

$1585. For further information 

Institute on Applied Rehab. Technolo©^, 
The Children’s Hospital, Fegan Plaza, 300 
: Longwood Avenue, Boston, Ma 021 15 

! 2. ACN does not list upcoming 
' events. I'm making an exception * 
;i because the topic is so pertinent j 
{ to this issue. A conference on •] 

J Adaptive Play and Microcomputers ; 
will be held in Asheville, NC, Sep- 
1 tember 21-22, 1989. Musselwhite ; 

; will share her creative approaches i 
; & expertise in a fun and energetic ’ 

■. fashion. For information contact i 

7 Irene Wortham Center, P.O. Box 5655, ; 

J Asheville, NC 28813 * 

13 Burkhart, L has 3 excellent books full of 
ideas and instructions for making 
homemade technology and app lying it crea- 
tively to enhance learning, 8503 Rhode Is- 
land Avenue, College Park, MD 20740. 

14 Lahm, E. (1987). Software designed to 
teach young multiply handicappeachildren 
to use the computer for controlling their en- 
vironment: A validation study. Unpublished 
doctoral dissertation: George Mason 
University, Fairfax, VA. 

15 Cook, A., Hussey, S., & Murphy, J. 

(1988). Using technology in a diagnostic- 
therapeutic paradigm for severely disabled 
clients. Paper presented at ISSAC Con- 
ference: Anaheim. 

16 Collier, B., Blackstcin-Adlcr, S. Thomas, 

D. (1988). Visual functional issues in AAC- 
Clinicat observations and implications. > 

ISAAC Conference: Anaheim. 3 



Resources 

(Call these people about visual scanning. 
They have given it lots of thought!) 

Jennifer Angelo, University at Buffalo, NY 
(716) 831-3141. 

Peggy Barker, Stanford Rehab.Eng. Center, 
CA? 415) 853-3345. 

Roxanne Butterfield. Don Johnson Develop- 
mental Equipment, IL (312) 526-2682. 

AJ Cook, Assistive Device Center, CA (916) 
278-6442. 

John Costello & Maggie Sauer, Communica- 
tion Enhancement Center, MA (617) 735- 
8392. V 

Cynthia Cress. Trace Research & Develop- 
ment Center, WI (608) 262-6966. 

Carol Goossens’, AC Service, U.C.P. of 
Greater Birmingham, AL (205) 251-0165. 

Ina Kirstein. Oakland Public Schools, Pon- 
tiac, MI (313) 858-1901. 

Robert Koch. 641 So. Palisade Drive, Orem, 
UT (801 >226-7997. 

Arlene Kjraat, Queens College, NY (718) 
520-7358. 

Janice Light. Hugh MacMillan Center, On- 
tario, CANADA (416) 425-6220. 

Judy McDonald & Paul Schwedja. Adaptive 
Peripherals, Inc., WA (206) 633-2610. 

Michael Palin, ACS Distributor, CA (707) 
864-6321. 

I^Procter, Private practice, CA (408) 336- 

Ann Ratcliff. Loma Linda University, CA 
(714) 824-4998. 

Gail VanTatenhove, Private practice, FL 
(407) 876-3423. . 

Walt Woltosz, Words 4-, CA (805) 949-833L 

Larry Weiss, Zygo Industries, OR (503) 684- 



Christine Wright, Stanford Rehab. Enz, Cen- 
ter, CA (415) 853-3345. 
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UPFRONT 

As we settle jnto the 1990s, lan- 
guage and cognitive issues are 
being revisited (e.g., literacy, lan- 
guage learning! relearning and the 
cognitive load of using augmentative 
techniques.) This issue is about 
graphic symbols. It is also about 
rethinking some basic assumptions. 

Many, many thanks to the profes- 
sionals I interviewed (see list of 
resources and references ). Their in- 
sights, willingness to engage in long 
discussions and to share their re- 
search and experiences were invalu- 
able! There are many unanswered 
questions about the use of graphic 
symbols in AAC. However, an army 
of researchers ( small, but powerful) 
will help us find some answers. In 



the meantime, we’ll need to be 
cautious as we make clinical 
decisions for and with people 
whose method of representing lan- 
gauge is through graphics. 

In For Consumers, research 
findings are shared and major is- 
sues are raised. The Equipment sec- 
tion highlights the need for profes- 
sionals to be familiar with their 
"tools” (i.e., commercially available 
graphic symbol sets and systems). 
You’ll also read about some new 
developments. In Clinical News the 
processes clinicians face in decision- 
making are discussed. For the 
Governmental section we shift 
gears and focus on money . Univer- 
sity & Research highlights exciting 
work in the Netherlands at the 

( continued on page 2) 



We are besieged by graphics. 
Graphics (pictures, signs, charts, 
logos, computer programs) inform, 
educate, influence and entertain us 
daily. Whereas artists, photogra- 
phers, and advertisers use graphic 
symbols as a medium for expres- 
sion, individuals with severe expres- 
sive communication disorders use 
graphic symbol sets and systems in 
qualitatively and quantitatively dif- 
ferent ways, i.e., as a primary means 
for language representation and in- 
teraction. Graphic symbols may: 

• 1. Stand for an individual’s internal repre- 
sentation of the environment 

• 2. Help individuals refine their knowledge 
of the world and influence their develop- 
ment of communication competence, cog- 
nition, language, and use of technology 

• 3. Serve as medium of expression 

• 4. Provide a window through which 
families/teachers/clinicians can leam 
about how individuals who don’t speak, 
organize their experiences conceptually 
and linguistically. 

As a field, we face major, un- 
answered questions about the 
processes connected with the per- 
ception . comprehension and use. of 
photos, pictures, and commercially 
available graphic symbol sets and 
systems. Many feel it is time to 
refocus on these issues. After all, it 
is one’s knowledge about the world 
and ability to express it that allows 
communication to occur. 

Studies reveal visual perceptual 
skills develop early, but mature use 
of graphic symbols is not observed 
until ages 7 or 8 years when child- 
ren use them to read, write, do 
math, etc. 

• At 3 months babies can discriminate their 
mother’s face from a stranger and be- 
tween 2 strangers. 

• By 6 months infants can "transfer" from 
live faces to highly realistic repre- 
sentations (color photographs) and 
"prefer" happy to angry or neutral faces. 

(continued on page 2) 
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Upfront 
(cont.) 

Institute 
for Rehabilita 
tion Research 
(IRV). 

Spring and/or 
Fall will soon 
be here (de- 
pending on your hemisphere)! I hope 
you’ll take time to smell the flowers! 
Remember, The Hotline Number is 
(408) 649-3050. 

Sarah Blackstone, Ph.D , Author, ACN 



For Consumers (cont from pg. 1) 

• By 12-18 months infants demonstrate com- 
prehension of photos/pictures paired with 
spoken words. 

• 24 month old infants discriminate between 
real and pictured objects. Only after 28 
months, can they use pictorial repre- 
sentations functionally. 

• Between 2 and 7 years children make per- 
ceptual judgments about how things look, 
focusing on 1 variable at a time. 

• 3 to 6year old children can imitate a live 
model significantly better than a photo, a 
doll, or a line drawing. However, many 3 
year olds have difficulty labeling actions of 
line drawings. (Mineo, review paper) 

Research within AAC is address- 
ing many issues directly related to 
graphic symbols. Table I sum- 
marizes some studies involving non- 
disabled adults and children as sub- 
jects. Questions include how easily 
symbols from various sets/systems 
are guessed, what perceptual and 
linguistic features may contribute to 
iconicity, and how adults and 
children approach graphic symbols. 
Table II, on the other hand, sum- 
marizes studies carried out with in- 
dividuals who use, or are potential 
users, of AAC symbols to communi- 
cate. These studies begin asking 
what learning processes are in- 
volved for targeted groups to com- 
prehend and use graphic symbols. 

Issues related to graphic symbols 
are complex in AAC , but we’ve 
clearly made a start. For example, 
we have learned that : 



Table I. Studies of AAC symbols with normal children and adults 

Author _ Subjects Symbols Task Result 



Yovetich Adults 
& Young 



uller & Adults 

Lloyd(1987) 

Lufllg & Adults 

Bcrsani(1985) 

Fuller Adults/ 

(1988) Children 



M usse tw h ileCh i Id re n 
R usee llo Adults 

(1984) 

ft aghavendra Children 

& Frisloe 

(1990) 

Bloomberg Adults 
(1984) 

Ecklund & Children 

Reichle 

(1987) 

LcPaee Children 

Mills(submitted) 



Blissymbols 

Blissymbols 

Blissymbols 


Guess meaning of 
symbols for 
concrete concepts 
Rate 

according to 
complexity 


Blissymbols 


Taught to label 
or point to 
symbols 


Blissymbols 

Picsyms 


Recognition of 
symbols 


Blissymbols 

standard/ 

enhanced 

Bliss, Picsyms 
PCS, PIC, 
Rebus 
Blissymbols 
Rebus 


Guess meaning 
10 actions, 

10 attributes 
10 entities 
Rated translucency 
nouns, verbs, 
modifiers 
Recall 15 symbols 
after repeated 
exposures 


PCS 


Used during 
reading instruction 



m 



Degree to which symbol re present- 
concept affects learning '] 

Best predictors arc: ! 

# semantic elements(comprchension) ; 

# of strokes ( perceptual) j 

Translucency better predictor of f 
learning than component complexity, j 
Adults learned more. Complexity ; 
influenced children's learning in j 
positive way. Adult data generalized : 
to nondisabled children. Learned j 
superimposed symbols easily \ 

Performance improves with age ) 
Picsyms & Rebus more guessable. j 

Identified perceptual features, letters * 
shapes, numbers. Related to objects i 
re: experience & world knowledge 
Focused on parts. Beginning reading { 
Rebus most translucent across 3 j 
linguistic classes j 

Performed well. Rebus recalled some- i 
what better. Children who pointed • 
remembered more than those wno i 
used verbal labels » 

Positive effect on attitudes toward j 
reading } 



| Table II. Studies of AAC symbols with special populations 

! Aulhor Subjects Findings 

For nouns, Picsyms, PCS more readily associated with spoken words, than 
Bliss. For verbs and descriptors, no significant differences 



| Mizuko * 
i & Reichle 

: | & Locke 
:! (1989) 

•j Hurlbut, 
IwaLa, & 
Green 
(1982) 
Sevcik & 
Romski 



Adults 

SMR 



some PH 
Severe 
physical/ 
cognitive 
handicaps 
Children 
Adoles. 
SMR 
Adoles. 
SMR 
Adults 
SMR 
Im-£s tab rooks 
Fitzpatrick Adults 
Barresi w/aphasia 



Children Compared transparency of 11 syml 
MMR-SMR suggested a hierarchy of difficulty 



bol types representing objects. Results 



Taught symbols representing known objects. Labels for iconic line drawin 1 
acquired more rapidly, maintained longer, generalized more frequently t 
labels for Blissymbols 



't 



itxon 
(1981) 
Mineo 



Learned arbitrary symbols as well as iconic symbols Those with 
nonfunctional language had more difficulty with line drawings 
than objects or photos. 

Those who failed to match objects with photo may have had problems 
with perceptual properties. Using cut-out figures facilitated matching , 

Matcn to sample paradigm within and between picture representation. Errori 
analysis suggests size is a crucial feature. 

Description of Visual Action Therapy (e.g., progression from tracing, 
picture, matching, picture commands, gest rues) r~* '— j -- * ■" 



II 



improvement 



I resulted in significant 



t & Children 

Mizuko(1989)MMR 
Pecyna Child 

(1988) w/ Downs 



Use of graphic mode via VOIS 136 increased when aided language- 
stimulation was provided 

Developmental level and level of communicative function mav influence 
effectivness of instructional program to teach graphic symbols 



ity, figure/ ground, and size, are im- 
portant considerations . 

2. While visual perception! dis- 
crimination influences symbol learn- 
ing it is an individual's ability to re- 
late perceptually-based information 
to their experiences and knowledge 
of things, ideas, relationships or 
events that is most important . 



ready known, spoken word with a 
symbol 

4 . Iconic symbols (i.e., those easily 
guessed) are 'easier to le am." How- 
ever, most "iconic* symbols repre- 
sent concrete objects/entities . Depict- 
ing actions, location, attributes, more 
abstract nouns, etc . requires the use 
of more abstract graphic symbols . 



O 

ERIC 



L Perceptual features can affect an 5. Translucency seems to have a posi- 

individuaVs ability to attach meaning live effect on learning when in- 
to symbols . Color, shape, complex- dividuals are learning to pair an al - 


5. We need to take into account the 
learning styles of individuals. Dif- 
ferent models for introducing and 
using graphic symbols with different 
populations are likely to emerge . For 
example, teaching symbols to normal 
talking individuals is different than 
(continued on page 3) 
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